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FOREWORD 


The  Army  Research  Office,  Office  of  the  Chief  of  Research  and 
Development,  Department  of  the  Army,  served  as  host  for  the  Tenth 
Conference  on  Design  of  Experiments  in  Army  Research,  Development 
and  Testing.  The  Conference  was  held  in  Washington,  D,  C.  during 
4-6  November  1964. 

The  continued  success  of  these  conferences  is  a  tribute  to  the 
foresight  of  Professor  Samuel  S,  Wilks  who  conceived  the  idea  of 
holding  such  conferences  and  chaired  the  Program  Committee  for  the 
first  nine  conferences.  Unfortunately,  due  to  hie  untimely  death, 
Professor  Wilks  could  not  participate  in  this  Tenth  Anniversary  Confer¬ 
ence.  His  effort  in  connection  with  these  Conferences  was  only  one  of 
Professor  Wilks'  many  contributions  to  the  Army.  His  wise  counsel 
and  advice  will  be  missed.  As  a  small  recognition  for  his  services  to 
the  Army,  this  Tenth  Anniversary  Conference  was  dedicated  to  the 
memory  of  Professor  Wilks. 

Almost  300  statisticians,  engineers  and  physicists  from  the  Army, 
other  government  agencies,  Army  contractors,  and  universities 
attended  the  conference.  This  number  far  exceeds  the  attendance  at 
any  of  the  previous  conferences  and  reflects,  in  part,  the  esteem  for 
Professor  Wilks  in  the  statistical  community. 

One  surprising  feature  was  the  announcement  that  Mr.  Philip  G. 

Rust  of  Thomasville,  Georgia,  had  contributed  funds  for  a  Samuel  S. 

Wilks  Award  to  be  presented  annually  at  the  Design  of  Experiments 
Conference.  It  is  especially  gratifying  that  a  long-time  civilian  employee 
of  the  U.  S.  Army,  Dr.  Frank  Grubbs,  Associate  Technical  Director  of 
the  Ballistic  Research  Laboratories,  was  the  recipient  of  the  initial 
award.  We  are  appreciative  that  the  American  Statistical  Association 
has  accepted  the  responsibility  for  determining  future  Award  winners. 

Because  of  the  particular  significance  of  this  Tenth  Conference, 
the  Program  Committee  invited  several  distinguished  statisticians  to 
deliver  papers:  Professor  H.  O.  Hartley,  Professor  Oscar  Kempthorne, 
Dr.  M  G.  Kendall  and  Professor  JohnW.  Tukey.  Professor  Gerald  J. 
Liebeiman  served  as  chairman  of  the  Panel  Discussion  on  Regression 
Analysis  and  arranged  for  Professor  G.E.  P.  Box,  Professor  Jack  C. 
Kiefer,  and  Professor  Ingram  Olkin  to  give  pertinent  papers  and  for 
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Professor  Robert  Bechhofer  to  serve  as  the  invited  discussant.  In 
addition  to  these  invited  addresses,  i3  papers  weie  ^ivor.  in  the  Clinical 
Sessions  and  18  papers  in  the  Technical  Sessions,  Additional  highlights 
of  the  meetings  were  the  after  dinner  presentations  by  Dr.  Churchill 
Eisenhart  and  Dr.  W.  J.  Youden. 

It  is  fitting  to  give  recognition  for  the  particular  activities  of  two 
groups  with  regard  to  these  Conferences.  The  Army  Mathematics 
Steering  Committee  (AMSC),  currently  chaired  by  Dr.  I.  R.  Hershner.Jr; 
is  commended  for  its  strong  support  of  these  Conferences  because  of  the 
actual  and  potential  gains  obtained  by  Army  facilities.  The  members  of 
the  Tenth  Conference  Program  Committee  are  commended  for  their 
work  in  obtaining  speakers,  selecting  a  location  and  planning  the  overall 
program.  The  members  of  this  Committee  were;  Dr,  F.  G.  Dressel 
(Secretary),  Mr.  Fred  Friahman,  Dr.  Walter  D.  Foster,  Dr.  FrankE. 
Grubbs  (Chairman),  Professor  Boyd  Harshbarger,  Professor  H.  L. 
Lucas,  Dr.  Clifford  .T.  Maloney,  Professor  Henry  B.  Mann  and  Professor 
Geoffrey  S.  Watson.  Special  credit  is  given  to  Dr.  F.  G.  Dressel  for 
performing  all  of  the  necessary  details  regarding  the  program,  invita¬ 
tions  and  the  publication  of  these  Proceedings. 

It  is  planned  to  have  an  Eleventh  Conference  at  Picatinny  Arsenal 
in  1965.  As  is  well  known,  these  Conferences  have  been  held  to  assist 
Army  statisticians  and  their  parent  organizations.  It  is  hoped  that 
Army  statisticians  will  continue  to  support  these  conferences  both  by 
the  presentation  of  scientific  papers  and  by  their  attendance. 


WALTER  E.  LOTZ,  JR. 
Director  of  Army  Research 


TENTH  CONFERENCE  ON  THE  DESIGN  OF  EXPERIMENTS 
IN  ARMY  RESEARCH,  DEVELOPMENT  AND  TESTING 


*  L  „  u  *  _  1  A£/1 


Wedneaday,  4  November 

0800-0900  REGISTRATION  --  Mezzanine  Floor  in  Foyer  No.  3  of  the 

Statler -Hilton  Hotel 

0900-0920  CALLING  OF  CONFERENCE  TO  ORDER  --  South  American 
Room,  Fred  Friahman,  Chairman  on  Local  Arrangements 

0920-1200  GENERAL  SESSION  1 

Chairman:  Major  General  Austin  W.  Betts,  Deputy  Chief  of 
Research  and  Development 

THE  STIMULUS  OF  S.  S.  WILKS  TO  ARMY  STATISTICS 
Major  General  Leslie  E.  Simon  (Ret'd),  Winter  Park,  Florida 

THE  SAMUEL  S.  WILKS  AWARD 

Announcement:  Don  Riley,  American  Statistical  Association 

Presentation:  Philip  G.  Rust,  Thomasville,  Georgia 

BREAK 

DEVELOPMENT  OF  THE  DESIGN  OF  EXPERIMENTS  OVER 
THE  PAST  TEN  YEARS 

Professor  Oscar  Kempthorne,  Iowa  State  University,'  Ames, 
Iowa 

1200  -1320  LUNCH 

Technical  Sessions  I  and  II  and  Clinical  Session  A  will  start  at  1320  and 
run  to  1500.  After  a  break  Technical  Sessions  III  and  IV  and  Clinical  Session 
B  will  convene  at  1540  and  run  to  1710, 
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1320  -  1500  TECHNICAL  SESSION  I  --  New  York  Room 

r1  Wo  »  »  a"  *  W  U  r  tiro  -y*  f  T?  r'BoOrrVt  a  nrl  «inn 

Army  Missile  Command,  Redstone  Arsenal,  Alabama 

APPLICATION  OF  DIMENSION  THEORY  TO  MULTIPLE 
REGRESSION  ANALYSIS 

David  R.  Howes,  U.  S.  Army  Strategy  and  Tactics 
Analysis  Group,  Bethesda,  Maryland 

THE  USE  OF  REGRESSION  ANALYSIS  FOR  CORRECTING 
OF  MATRIX  EFFECTS  IN  THE  X-RAY  FLUORESCENCE 
AN  LYSES  OF  PYROTECHNIC  COMPOSITIONS 
R.  H.  Myers  and  B.  J.  Alley,  Virginia  Polytechnic  Institute, 
Blacksburg,  Virginia,  Rep.  Redstone  Arsenal 

1320  -1500  TECHNICAL  SESSION  II  -  South  American  Room 

Chairman:  Henry  Ellner,  Directorate  for  Quality  Assurance, 
Edgewood  Arsenal,  Maryland 

SAMPLING  FOR  DESTRUCTIVE  OR  EXPENSIVE  TESTING 
Joseph  Mandelson,  Directorate  of  Quality  Assurance, 

U.  S.  Army  Edgewood  Arsenal,  Edgewood  Arsenal,  Md. 

TOTAL  SAMPLE  STATISTICS  FROM  SUBSAMPLE  STATISTICS 
Paul  C.  Cox,  Reliability  and  Statistics  Office,  Army  Missile 
Test  and  Evaluation  Directorate,  White  Sands  Missile  Range, 
New  Mexico 

1320  -1500  CLINICAL  SESSION  A  -  -  California  Room 

Chairman:  Ira  A.  DeArmon,  Jr.,  Operations  Research  Group, 
Army  Chemical  Corps,  Edgewood  Arsenal,  Md. 


Panelists: 

Dr.  Frank  E,  Grubbs,  Army  Ballistic  Research  Laboratories, 
Aberdeen  Proving  Ground,  Maryland 

Professor  H.  C.  Hartley,  Institute  of  Statistics,  Agricultural 
and  Mechancial  College,  College  Station,  Texas 
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Panelists  (cont'd): 

Dj.  Emil  H.  Jebe,  Institute  of  Science  and  Technology, 

The  University  of  Michigan,  Ann  Arbor,  Michigan 

Professor  Gerald  J.  Lieberman,  Stanford  University, 
Stanford,  California 

Professor  H.  L.  Lucas,  institute  of  Statistics,  North 
Carolina  State  of  the  U.N.C.  ,  Raleigh,  North  Carolina 

SYSTEM  CONFIGURATION  PROBLEMS  AND  ERROR 
SEPARATION  PROBLEMS 
Fred  S.  Hanson,  Plan  and  Operations  Directorate, 

White  Sands  Missile  Range,  New  Mexico 

AN  EXPERIMENT  IN  MAKING  TECHNICAL  DECISIONS  USING 
OPERATIONS  RESEARCH  AND  STATISTICAL  METHODS 
Andrew  H,  Jenkins,  U.  S.  Army  Missile  Command, 
Huntsville,  Alabama,  and  Edwin  M.  Bartee,  School  of 
Engineering,  University  of  Alabama 

1500-1540  BREAK 

1540-1710  TECHNICAL  SESSION  III  --  New  York  Room 

Chairman:  Morris  A,  Rhian,  Operations  Research  Group, 
Army  Chemical  Corps,  Edgewood  Arsenal,  Md. 

IMPROVEMENT  CURVES:  PRINCIPLES  AND  PRACTICES 
Jerome  H.  N,  Selman,  Stevens  Institute  of  Technology, 

Rep,  the  U.  S.  Army  Munitions  Command,  Dover,  N.  J. 

THE  EFFECT  OF  VALIDITY,  LENGTH,  AND  SCORE 
CONVERSION  ON  A  MEASURE  OF  PERSONNEL  ALLOCATION 
EFFICIENCY 

Richard  C.  Sorenson  and  Cecil  D.  Johnson,  U.  S.  Army 
Personnel  Research  Office,  Washington,  D.  C. 


vi 


1540  - 1710  TECHNICAL  SESSION  IV  --  South  American  Room 

^  .  ■  ......  t  i  t"»  t  nr»  »  _  1  .  *  .  t  *  *— *. 

uuaii  man,  juoc^u  x\ »  Lane  *  a  ctiuuv.at  xl  vaiuatiun  umtC  , 

Army  Research  Office -Durham,  Durham,  N.  C. 

A  QUANTITATIVE  ASSAY  FOR  CRUDE  ANTHRAX  TOXINS 
Bertram  W.  Haines,  U.  S.  Army  Biological  Labs.  ,  Fort 
Detrick,  Frederick  Maryland 

AN  INVESTIGATION  OF  THE  DISTRIBUTION  OF  DIRECT 
HITS  ON  PERSONNEL  BY  SELF -DISPERSING  BOMBLETS 
David  M.  Moss  and  Theodore  W.  Horner,  Booz-Allen 
Applied  Research,  Inc.  ,  Bethesda  14,  Maryland 
Rep.  Biomathematics  Division  of  Fort  Detrick,  Maryland 

1540  -1710  CLINICAL  SESSION  B  --  California  Room 

Chairman:  Henry  A.  Dihm,  Advanced  Systems  Laboratory, 
Army  Missile  Command,  Redstone  Arsenal,  Alabama 

Panelists: 

Dr,  O.  P.  Bruno,  Surveillance  Group,  Army  Ballistics 
Research  Laboratories,  Aberdeen  Proving  Ground,  Md. 

Dr.  Donald  S.  Burdick,  Duke  University,  Durham,  N.  C. 

Professor  Clyde  Y.  Kramer,  Virginia  Polytechnic  Institute, 
Blacksburg,  Virginia 

Dr.  R.  L.  Stearman,  C-E-I-R,  Inc.  ,  Los  Angeles,  Calif. 

Dr.  William  Wolman,  National  Aeronautics  and  Space 
Administration,  Goddard  Space  Flight  Center,  Greenbelt, 
Maryland 

EXPLOSIVE  SAFETY  AND  RELIABILITY  ESTIMATES  FROM 
A  LIMITED  SIZE  SAMPLE 

J  .  N.  Ayres,  L.  D.  Hampton  and  I.  Kabik,  U,  S.  Naval 
Ordnance  Laboratory,  White  Oak,  Silver  Spring,  Maryland 
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COMPARING  THE  VARIABILITIES  OF  TWO  TEST  METHODS 
USING  DATA  KOR  SEVERAL  POPULATIONS 
Manfred  W.  Krimmer,  U.  S.  Army  Ammunition  Procurement 
and  Supply  Agency,  Joliet,  Illinois 


Thursday,  5  November 

Technical  Session  V  and  Clinical  Session  C  and  D  will  run  from  0830-1010. 
After  the  break  General  Session  2  will  convene  at  1050.  After  lunch  Technical 
Sessions  VI  and  VII  and  Clinical  Session  E  will  start  at  1300  and  end  at  1420. 
The  Panel  Discussion  is  scheduled  to  be  conducted  from  1450  to  1710.  Follow¬ 
ing  the  banquet,  which  starts  at  1900,  there  will  be  two  short  talks. 

0830-1010  TECHNICAL  SESSION  V  --  South  American  Room 

Chairman:  R.  H.  Myers,  Statistical  Laboratory,  Virginia 
Polytechnic  Institute,  Blacksburg,  Virginia 

CYCLIC  DESIGNS 

H.  A.  David  and  F.  W.  Wolock,  University  of  North  Carolina 
and  Virginia  Polytechnic  Institute,  Rep.  Army  Research  Office- 
Durham 

SOME  RESULTS  ON  THE  FOUNDATIONS  OF  STATISTICAL 
DECISION  THEORY 

Bernard  Harris,  J.  D.  Church,  F.  V.  Atkinson, 

Mathematics  Research  Center ,  U.  S.  Army,  University  of 
Wisconsin,  Madison,  Wisconsin 

0830-  1010  CLINICAL  SESSION  C  -*  California  Room 

Chairman:  Dr.  Erwin  L.  LeClerg,  Biometrical  Services 
Division,  U.  S.  Department  of  Agriculture,  Plant  Industry, 
Beltsville,  Maryland 

« 

Panelists: 

Dr.  Walter  D.  Foster,  Biometrics  Division,  Army 
Biological  Warfare  Laboratories,  Fort  Detrick,  Md. 

Dr.  Samuel  W.  Greenhouse,  Biometrics  Branch,  National 
Institute  of  Mental  Health,  Bethesda,  Maryland 
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Panelists  (cont'd): 

Professor  Clyde  Y.  Kramer,  Virginia  Polytechnic  Institute, 
Blacksburg,  Virginia 

Professor  H.  L,  Lucas,  North  Carolina  State  of  the  UNC, 
Raleigh,  North  Carolina 

Dr.  Clifford  J.  Maloney,  Division  of  Biologies  Standards, 
National  Institutes  of  Health,  Bethesda,  Maryland 

DISINFECTION  OF  AEROSOLIZED  PATHOGENIC  FUNGI  ON 
LABORATORY  SURFACES 

Richard  H.  Kruse,  Theron  D.  Green,  Richard  C.  Chambers 
and  Marian  W.  Jones,  U.  S.  Army  Biological  Laboratories, 
Fort  Detrick,  Frederick,  Maryland 

THE  EFFECT  OF  SNAKE  VENOM  AND  ENDOTOXIN  ON 
CORTICAL  ELECTRICAL  ACTIVITY 
James  A.  Vick,  Henry  P.  Ciuchta,  Edward  H.  Polley, 
and  James  Manthei,  Directorate  of  Medical  Research, 
Chemical  Research  and  Development  Laboratories, 

Edgewood  Arsenal,  Maryland 

COMPUTER  ANALYSIS  OF  RHESUS  MONKEY  IN  VISUAL 
DISCRIMINATION  TESTING 
JohnC.  Atkinson,  Directorate  of  Medical  Research, 
Chemical  Research  and  Development  Laboratories, 

Edgewood  Arsenal,  Maryland 

CLINICAL  SESSION  D  --  New  York  Room 

Chairman:  Lee  W.  Green,  Jr.,  Florida  Research  and 
Development  Center,  Pratt  and  Whitney  Aircraft,  West 
Palm  Beach,  Florida 

Panelists; 

Professor  R.  E.  Bechhofer,  Cornell  University, 

Ithaca,  New  York 

Professor  G.  E,  P.  Dox,  the  University  of  Wisconsin, 
Madison,  Wisconsin 
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Professor  G.  J.  Lieberman,  Stanford  University, 

Stanford,  California 

Dr.  H,  B.  Mann,  Mathematics  Research  Center ,  U.  S. 

Army,  University  of  Wisconsin,  Madison,  Wisconsin 

FATIGUE  -  LIMIT  ANALYSES  AND  DESIGN  OF  FATIGUE 
EXPERIMENTS 

A.  H.  Soni  and  R.  E.  Little,  Oklahoma  State  University, 
Stillwater,  Oklahoma,  Representing  Army  Research  Office- 
Durham 

GETTING  REGRESSION  ANALYSIS  IMPLEMENTED 
W.  H.  Ammann,  U.  S.  Army  Aviation  Materiel  Command, 

St.  Louis,  Missouri 

1010  -1050  BREAK 

1050  -1150  GENERAL  SESSION  2  --  South  American  Room 

Chairman:  Dr.  Walter  D.  Foster,  Biometric  Div.  ,  Army 
Biological  Warfare  Labs.  ,  Fort  Detrick,  Frederick,  Md. 

ASSESSMENT  AND  CORRECTION  OF  DEFICIENCES  IN  PERT 
Drs.  H,  O.  Hartley  and  A.  W.  Wortham,  Institute  of 
Statistics,  Texas  A  and  M  University,  College  Station,  Texas 

1150  -1300  LUNCH 

1300  -1420  TECHNICAL  SESSION  VI  --  South  American  Room 

Chairman:  Leonard  Pepper,  Concrete  Division,  U.  S.  Army 
Engineer  Waterways  Experiments  Station,  Vicksburg,  Miss. 

TEQUILAP:  TEN  QUANTITATIVE  ILLUSIONS  OF 
ADMINISTRATIVE  PRACTICE 
Clifford  J.  Maloney 
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COMBAT  VEHICLE  FLEET  MANAGEMENT 
C.  J.  Christian  ccr.  and  Mr.  G.  F.  r.nnn»r^  P#*B*arrh 
Analysis  Corporation,  McLean,  Virginia 

1300  -  1420  TECHNICAL  SESSION  VII  --  New  York  Room 

Chairman:  Eugene  F.  Smith,  Concrete  Division,  U.  S. 

Army  Waterways  Experiment  Station,  Vicksburg,  Miss. 

APPLICATION  OF  STATISTICS  TO  EVALUATE  SWIVEL 
HOOK  TYPE  CROSS  CHAIN  FASTENERS  FOR  MILITARY 
APPLICATIONS  OF  TIRE  CHAINS 
Otto  H.  Pfeiffer,  Components  Research  and  Development 
Labs.  ,  Army  Tank-Automotive  Center,  Warren,  Michigan 

SOME  FACTORS  AFFECTING  THE  PRECISION  OF  CO-ORDINATE 
MEASUREMENTS  ON  PHOTOGENIC  PLATES 
Desmond  O'Connor,  Research  and  Analysis  Division,  U.  S. 

Army  Engineer  Geodesy,  Intelligence  and  Mapping  Research 
and  Development  Agency,  Fort  Belvoir,  Virginia 

1300  -1420  CLINICAL  SESSION  E  --  California  Room 

Chairman:  Joseph  Mandelson,  Directorate  of  Quality 
Assurance,  Edgewood  Arsenal,  Maryland 

Panelists; 

Professor  Donald  S.  Burdick,  Duke  University, 

Durham,  North  Carolina 

Dr.  Bernard  Harris,  Mathematics  Research  Center, 

U.  S.  Army,  University  of  Wisconsin,  Madison,  Wis. 

Professor  Ingram  Olkin,  Stanford  University,  Stanford, 

California  ; 

Dr.  H.  M.  Rosenblatt,  Statistical  Research  Division, 

Bureau  of  the  Census,  Washington,  D.  C. 

Professor  G.  S.  Watson,  The  Johns  Hopkins  University, 

Baltimore,  Maryland 
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ERROR  ANALYSIS  PROBLEMS  IN  THE  ESTIMATION  OF 
SPECTRA 

Virginia  Tipton,  Plans  and  Operations  Directorate, 

White  Sands  Missile  Range,  New  Mexico 

VALIDATION  PROBLEMS  OF  AN  INTERFERENCE 
PREDICTION  MODEL 

William  B.  McIntosh,  Army  Electronics  Proving  Ground, 
Fort  Huachuca,  Arizona 

1420  -1450  BREAK 

1450  -1710  GEN  ERAL  SESSION  3  --  South  American  Room 

PANEL  DISCUSSION  ON  REGRESSION  ANALYSIS 
Chairman:  Professor  Gerald  J.  Lieberman, 

Stanford  University 

Panelists  and  the  Titles  of  their  Addresses: 

USE  AND  ABUSE  OF  REGRESSION 
Professor  G.  E.  P,  Box,  The  Uni versity  of  Wisconsin 

OPTIMUM  EXTRAPOLATION  AND  INTERPOLATION 
DESIGNS 

Professor  JackC.  Kiefer,  Cornell  University 

ESTIMATION  FOR  A  REGRESSION  MODEL  WITH 
CON  VARIANCE 

Professor  Ingram  Olkin,  Stanford  University 
Discussant:  Professor  Robert  Bechhofer,  Cornell  University 
1900  BANQUET 

Evening  Session  Chairman:  Dr,  I.  R.  Hershner,  Jr.  ,  ARO 

SAM  WILKS  AS  I  REMEMBER  HIM 
Dr,  Churchill  Eisenhart,  National  Bureau  of  Standards, 
Washington,  D.  C. 

AN  OPERATIONS  RESEARCH  YARN  AND  OTHER  COMMENTS 
Dr.  W.  J.  Youden,  National  Bureau  of  Standards, 
Washington,  D.  C, 
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x  ctluuv«i  Sessions  V1V  and  IX  as  well  as  Clinical  Session  F  run  from 
0830  to  0950.  General  Session  4  will  start  at  1020  and  end  at  ix20. 

0830-0950  TECHNICAL  SESSION  VIII  --  South  American  Room 

Chairman:  Donald  S.  Burdick,  Duke  University,  Durham,  N.  C. 

THE  DESIGN  OF  COMPLEX  SENSITIVITY  EXPERIMENTS 
D.  Rothman  and  J,  M.  Zimmerman,  Mathematic  and 
Statistics  Group,  Rocketdyne,  A  Division  of  N.  American 
Aviation,  Canoga  Park,  Calif.  Rep.  George  C.  Marshall 
Space  Flight  Center,  NASA,  Huntsville,  Alabama 

FACTORS  AFFECTING  SENSITIVITY  EXPERIMENTS 
J.  R.  Kniss  and  W,  Wenger,  U.  S.  Army  Ballistic  Research 
Labs.  ,  Aberdeen  Proving  Ground,  Maryland 
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0830-0950  TECHNICAL  SESSION  IX  --  New  York  Room 

Chairman:  Ralph  E.  Brown,  U.  S.  Army  Munitions 
Command,  Philadelphia,  Pennsylvania 

A  COMPARISON  OF  RECONNAISSANCE  TECHNIQUES 
FOR  LIGHT  OBSERVATION  HELICOPTERS  AND  A 
GROUND  SCOUT  PLATOON 

Harrison  N.  Hoppea,  Barry  M,  Kibel,  Arthur  R.  Woods, 
Research  Analysis  Corporation,  McLean,  Virginia 

A  STUDY  OF  PROBABILITY  ASPECTS  OF  A  SIMULATANEOUS 
SHOCK  WAVE  PROBLEM 

Edward  C.  Hecht,  Nuclear  Engineering  Directorate, 

Picatinny  Arsenal,  Dover,  New  Jersey 

0830-0950  CLINICAL  SESSION  F  --  California  Room 

Chairman:  Dr.  B.  W.  Haines,  U.  S,  Army  Biological 
Laboratories,  Fort  Detrick,  Maryland 
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Professor  R.  E,  Bechhofer,  Cornell  Univeriity, 

Ithaca,  New  York 

Mr.  David  R.  Howe»,  U.  S.  Army  Strategy  and 
Tactics  Analysis  Group,  Bethesda,  Maryland 

Dr.  R.  J.  Lundegard,  Logistics  and  Mathematical 
Statistics  Branch,  Office  of  Naval  Research, 

Washington,  D.  C. 

Professor  Ingram  Olkin,  Stanford  University, 

Stanford,  California 

Professor  G.  S.  Watson,  The  Johns  Hopkins  University, 
Baltimore,  Maryland 

A  DATA  COLLECTION  PROCEDURE  FOR  ASSESSING  NEURO¬ 
MOTOR  PERFORMANCE  IN  THE  PRESENCE  OF  MISSILE 
WOUNDS 

William  H.  Kirby,  Jr.,  William  Kokinakis,  Larry  M. 
Sturdivan  and  William  P.  Johnson,  Ballistic  Research 
Aberdeen  Proving  Ground,  Maryland 

PROBLEMS  IN  THE  DESIGN  OF  STATISTICS-GENERATING 
WAR  GAMES 

William  H.  Sutherland,  Research  Analysis  Corporation, 
McLean,  Virginia 

0950  -1020  BREAK 

1020  -1220  GENERAL  SESSION  4  --  South  American  Room 

Chairman:  Dr.  Frank  E.  Grubbs,  Chairman  of  the 
Conference,  Ballistic  Research  Laboratories, 

Aberdeen  Proving  Ground,  Maryland 

THE  FUTURE  OF  PROCESSES  OF  DATA  ANALYSIS 
\  Professor  John  W,  Tukey,  Princeton  University, 

■  Princeton,  New  Jersey 
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TO  ARMY  STATISTICS 

Leslie  E.  Simon 
Major  General,  USA  (Ret. ) 

ABSTRACT  ■  The  stimulus  of  S.  S.  Wilks  to  the  scientific  community 
is  discussed  briefly,  followed  by  a  more  detailed  account  of  his  originating 
the  idea  oi  a  series  of  Army-wide  conferences  on  design  of  experiments  in 
Army  research,  development  and  testing.  The  Army's  rather  satisfactory 
progress  in  statistical  methodology  prior  to  the  conference  series  is  dis¬ 
cussed,  with  comments  on  its  limitations  and  less  than  ideal  direction  of 
procedure.  Wilks'  apparent  perception  of  the  situation,  his  courage  in 
undertaking  a  large  and  difficult  task,  and  his  surprisingly  large  measure 
of  success  is  discussed.  The  importance  of  carrying  on  the  spirit  of  Wilks 
is  emphasized,  and  the  creation  of  The  Wilks  Award,  as  a  measure  to  that 
end  is  mentioned. 

ORIGIN  OF  THE  CONFERENCE  SERIES.  Mr.  Chairman,  Fellow 
Conferees,  Ladies  and  Gentlemen,  Samuel  Stanley  Wilks  was  my  very  good 
friend  most  of  his  professional  life.  Whereas  1  am  aware  of  many  of  Wilks' 
dedicated  and  outstanding  services  at  a  national,  if  not  a  world  level,  I 
prefer  to  concentrate  my  remarks  on  an  area  of  Wilks'  career  that  is  close 
to  home  to  me:  the  very  valuable  services  that  he  did  voluntarily  for  the 
Army.  I  am  sure  that  others  more  able  than  I  will  cover  his  broader  serv¬ 
ices  as  a  teacher,  both  academic  and  extra  curricular;  as  a  research 
w  orker,  as  an  organizer,  and  as  a  competent  and  inspiring  leader.  Frederick 
Mosteller  has  presented  an  excellent  outline  of  Wilks'  worldwide  work  in  the 
April,  1964  issue  of  "The  American  Statistician",  under  the  title,  "Samuel 
S.  Wilks;  Statesman  of  Statistics".  Mosteller's  paper  should  serve  as  a 
guide  for  other  papers  on  Wilks.  However,  I  cannot  help  observing  that 
although  Mosteller's  title  is  justified,  I  hope  that  he  will  forgive  me  if  I 
observe  that  Wilks  was  by  his  own  choice  somewhat  lacking  in  the  formality 
associated  with  statesmanship.  Contrary  to  one's  concept  of  dignity,  Sam 
was  "just  folks",  whether  he  was  talking  with  a  first-rate  scientist,  a  neophyte 
In  Applied  Statistics  or  a  man  primarily  a  soldier.  He  knew  and  understood 
people;  and,  by  nature  was  ever-ready  to  give  any  help  within  his  competence 
to  anyone  who  genuinely  needed  it.  It  was  in  the  latter  two  capacities,  that 
I  had  my  entree  to  Wilks. 

It  v  as  over  fifteen  years  after  our  initial  meeting  that  Wilks  made  a 
proposal  that  has  helped  much  in  improving  Army  organization,  doctrine, 
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tactics  and  weapons;  and,  at  the  same  time  contributed  to  improving  the 
morale  of  Army  personnel,  and  to  saving  time  and  expense  iu  iiii.lil.iiy 
research  and  development. 

In  late  1954  or  early  1955,  when  I  was  Assistant  Chief  of  Ordnance  for 
Research  and  Development,  U.  S,  Army,  Wilks  proposed  that  the  Army 
establish  a  series  of  Army-wide  conferences  on  design  of  experiments  in 
Army  research,  development  and  testing.  Dr.  Frank  E.  Grubbs,  who, 
under  the  authority  of  my  office,  had  chaired  an  Ordnance  symposium  on 
Statistical  Methods  in  1953  [l]  ,  strongly  indorsed  Wilks'  proposal  for 
Army-wide  conferences,  devoted  primarily  to  design  of  experiments;  and, 
of  course,  I  concurred.  The  Army  Mathematics  Advisory  Panel*  (later, 
designated  as  the  Army  Mathematics  Steering  Committee)  operated  under 
the  Office  of  Ordnance  Research  (now  Army  Research  Office -Durham);  and 
consequently  the  responsibility  for  the  conferences  was  assigned  to  that 
office.  Wilks'  proposal  was  made  pursuant  to  a  survey  made  by  the  Army 
Mathematics  Steering  Committee  in  which  they  investigated  over  30  Army 
facilities.  They  found  that  one  of  the  most  frequently  mentioned  needs 
expressed  by  the  scientific  personnel  was  for  greater  knowledge  of  modern 
etatistical  theory  of  the  design  and  analysis  of  experiments.  The  First 
Conference  on  Design  of  Experiments,  in  Army  Research,  Development  and 
Testing  was  held  on  October  19-21,  1955  at  the  Diamond  Ordnance  Pubs 
Laboratories  and  The  National  Bureau  of  Standards.  Wilks  chaired  all  the 
conferences  up  to  the  present  Tenth  Conference.  f 

1  believe  that  observing  as  best  we  can  the  time -rate -of -change  of  the 
character  of  these  conferences  and  the  concurrent  increase  of  basic  under¬ 
standing  of  the  interrelationships  of  men,  weapons,  organization,  doctrine, 
tactics,  and  research  and  development,  will  throw  1.  rht  on  the  beneficial 
influence  of  Wilks  on  National  Defense.  I  do  not  mean  to  infer  that  all 
Statistical  progress  is  due  to  Wilks;  but  1  am  sure  that  much  of  the  progress 
is  due  to  the  spirit  of  cooperation  that  he  infused,  to  his  influence  and  to  his 


*The  Army  Mathematics  Advisory  Panel,  of  which  Wilks  was  a  member  was 
operated  by  the  Ordnance  Corps  for  the  Office  of  the  Chief  of  Research  and 
Development,  U.  S.  Army.  I  am  indebted  to  Colonel  P.  N.  Gillon  (Ret,  ), 
who  was  both  the  Commanding  Officer  of  the  Office  of  Ordnance  Research 
(Durham)  and  the  very  able  Chairman  of  the  Army  Mathematics  Advisory 
Panel  for  the  clear,  curt  minutes  and  records  that  he  left,  and  especially 
for  reference  [2]  . 
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personal  contributions.  Similarly.  I  believe  that  the  history  cf  W ilka  in  this 
relatively  small  sub-field  of  his  very  active  life  is  a  close  parallel  to  the 
fruitfulness  of  his  activity  in  other  fields  to  which  he  devotjd  far  more  time. 

Let  us,  then,  observe  the  status  of  Army  statistics  up  to  1953;  trace,  at 
least  approximately,  the  conferences  on  Design  of  Experiments  in  Army 
Research,  Development  and  Testing;  and  observe  the  present-day  status  of 
Army  statistics. 

Incidentally,  the  Army  was  neither  without  statistical  sophistication  in 
1953,  nor  is  its  knowledge  optimum  today. 

SUMMARY  OF  ARMY  STATISTICAL  PROGRESS,  BETWEEN  WORLD 
WAR  I  AND  II.  Historically,  the  application  of  probability  theory  to  the 
dispersion  of  shots  on  a  target  appears  to  be  about  the  only  Army  use  of 
Statistics,  prior  to  World  War  I.  There  was  c.  jump  in  mathematical  sophis¬ 
tication  during  World  War  I,  due  to  A.  A.  Bennett  [3]  ,  Fowler  [4]  ,  Moulton  [5]  , 
and  others  In  connection  with  progress  in  applying  statistics  to  Ballistic  prob¬ 
lems.  Between  World  Wars  I  and  II,.  Kent,  Dederick,  McShane  and  others 
developed  further  applications  of  Statistics  in  connection  with  Ballistics.  The 
staff  of  the  Bell  Telephone  Laboratories,  especially  Dr.  Walter  A.  Shewhart 
and  Harold  F.  Dodge,  was  most  fruitful  in  the  discovery  of  Statistical  techniques, 
and  the  Army  was  a  shameless  plagiarist  in  adapting  them  to  its  problems.  Shew- 
hart's  work  [6]  led  to  the  Army's  first  full-scale  industrial  use  of  Statistical 
Quality  Control  in  manufacture  at  Picatinny  Arsenal,  Dover,  New  Jersey,  which 
also  was  certainly  one  of  the  first  few  of  such  uses  in  the  world.  The  Army 
Ammunition  Surveillance  [7]  (Stockpile  Reliability)  System  (circa  1939)  was 
based  largely  on  what  was  very  recent  work  at  that  time.  The  Dodge-Romig 
Sampling  Tables,  not  yet  in  book  form  [8]  ,  appeared  just  in  time  for  use  for 
ammunition  inspection  ar.d  acceptance  tests  in  World  War  II.  During  the  period 
shortly  before  World  War  II,  the  Army  felt  a  bit  smug  about  itc  statistical 
competence. 

ARMY  STATISTICAL  PROGRESS  DURING  WORLD  WAR  II.  World  War 
II  saw  great  progress  in  the  military  use  of  Statistics,  due  primarily  to  the 
availability  to  the  war  effort  of  men  of  competence.  The  National  Defense 
Research  Council  (later,  Office  of  Scientific  Research  and  Development),  the 
staff  of  the  BRL,  and,  to  a  lesser  extent,  the  staffs  of  Ordnance  Arsenals 
acquired  many  Mathematicians  and  Statisticians  of  competence.  Procedures 
for  specifications  of  materiel,  sampling,  testing  and  interpretation  of  data 
(both  planned  data  and  the  salvag'ng  of  unplanned  data)  were  greatly  improved. 
Indeed,  Operations  Research  was  being  born  even  then.  The  Army#  was  not 
unmindful  of  the  possible  adaptation  of  any  new  Statistical  "tool"  to  its  work. 

^References  to  the  Army  do  not  imply  that  the  Navy  and  Air  Force  did  not 
also  make  progress, 
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In  addition  to  the  above  uses  of  Statistical  Methods  substantial  progress 
was  made  by  the  Army  during  World  War  II  of  which  th*r»  i«  little  cr  r. z 
record.  Many  new  techniques  such  as  Sequential  Sampling  and  Reliability 
were  actually  used  in  the  Army,  at  least  in  an  empirical  way,  before  they 
were  later  designated  by  appropriate  specific  names.  Of  course,  needed 
theory  was  not  worked  out  in  a  formal  way  at  that  time.  For  example,  the 
formal  presentation  of  sequential  sampling  had  to  await  the  work  of 
Dr.  Abraham  Wald,  which  was  not  published  in  book  form  until  1947  [9]  . 

ARMY  STATISTICAL  PROGRESS,  WORLD  WAR  II  -  1953.  After  World 
War  II,  progress  continued,  although  its  rate  was  diminished  due  both  to 
decrease  in  staff  and  to  loss  of  some  of  the  more  competent  people.  Appar¬ 
ently,  experiments  that  involved  Factorial  Designs  were  the  first  instance 3 
of  full  use  of  Experimental  Designs  in  the  Army.  Factorial  designs  were 
used  at  the  Ballistic  Research  Laboratories  in  the  study  of  armor  plate 
(1946-47)v,  in  the  mammoth  experiment  on  Aircraft  Vulnerability  (1946-50)* , 
and  even  on  Project  Stalk  (a  tank-fire  control  study  under  field  conditions)* 
circa  1953.  In  1953-1954  Reliability  (10]  ,  in  its  present  day  sense,  was  used 
by  Ordnance  Research  and  Development,  in  a  full-scale  organizational  and 
technical  way,  as  a  means  of  rescuing  the  Country's  first  operational  guided 
missile,  the  NIKE,  from  a  serious  threat  of  failure. 

With  this  rather  glowing  account  of  Army  progress  and  statusr  one 
might  well  question  wherein  was  the  Army  laggard,  and  where  was  the  fail¬ 
ure  or  potential  threat  of  failure?  What  great  work  was  there  left  to  be  done 
by  the  series  of  conferences  on  design  of  experiments  under  Wilks?  I  shall 
show  that  a  very  great  deal  was  wrong  with  the  Army's  use  (or  lack  of  use) 
of  statistical  methods;  that  the  task  of  righting  the  wrong  was  formidable, 
both  in  magnitude  and  in  potential  obstacles;  and  that  astonishing  progress 
has  been  made  on  the  task  during  the  nine  years  of  the  conferences. 

From  the  survey  of  the  30  Army  facilities,  Wilks  must  have  understood 
rather  well  what  the  Army  needed,  and  have  understood  also  the  need  for 
newly  organized  and  sustained  effort  to  supply  the  need.  His  skill  as  a 
teacher  must  have  fortified  him  from  fear  of  failure  in  undertaking  to  change 
the  mode  of  operation  of  a  large  segment  of  the  Army. 

’’'Ballistic  Research  Laboratories  Publications. 
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WHAT  WAS  WRONG.  Let  us  observe  that  the  origin,  growth,  and  use 
of  Statistical  Methods  in  the  Army  was  not  only  unplanned,  but  actually 
tended  to  progress  in  the  least  advantageous  direction;  i,  a.  ,  from  end¬ 
point  to  origin,  rather  than  from  origin  to  end.  Roughly  speaking,  we  can 
regard  the  military  regime  as  consisting  of  the  following  steps  or  stages; 
doctrine,  tactics,  organization,  selection  of  equipment,  fabrication  of 
equipment,  test  of  equipment,  and  use  of  equipment.  Logically,  a  power¬ 
ful  medium  for  the  improvement  of  a  stage  should  be  first  applied  to  the 
preceding  stage  or  stages  to  which  it  is  applicable.  For  example,  a  big 
improvement  in  use  of  equipment,  (e.  g.  ,  accuracy  of  ammunition)  loses 
much  of  its  potentially  beneficial  effect  if  either  the  tactics,  organization, 
or  weapons  system  is  poor. 

Contrary  to  the  above  observation,  the  earliest  use  of  probability 
theory  by  the  Army  was  for  use  of  equipment,  viz,  the  adjustment  of  artil¬ 
lery  fire.  The  use  of  techniques  based  on  the  Gaussian  Distribution,  or 
Normal  Probability  Law,  in  connection  with  artillery  fire  probably  is 
exceeded  in  antiquity  only  by  the  use  of  elementary  probability  theory  in 
connection  with  games  of  change  [ll ]  . 

Decades  elapsed  before  the  next  major  step.  In  1936,  the  Army  began 
to  use  Statistical  Quality  Control  in  the  manufacture  of  equipment,  vie,  the 
production  of  ammunition  at  Picatinny  Arsenal,  Dover,  New  Jersey.  Kindred 
techniques  such  as  sampling  theory  and  statistical  methods  for  analyzing 
data  soon  spread  to  improve  specifications,  inspections  and  acceptance  tests. 

During  World  War  II  almost  all  fabrication  of  military  equipment  was 
better,  cheaper,  and  quicker,  due  largely  to  these  techniques.  During 
World  War  II,  one  strange  reversal  occurred  in  the  inverse  order  of  progress. 
Operations  Research  was  born  out  of  military  sponsorship  and  was  actually 
used  to  a  limited  degree  by  the  staffs  of  high  military  planners  in  connection 
with  the  planning  of  the  operations  of  large  combat  forces. 

After  World  War  II,  it  began  to  be  more  and  more  realised  that  since 
Statistical  Methods  improved  the  quality  of  equipment  and  reduced  costs  it 
would  be  a  good  idea  to  use  similar  techniques  with  the  research,  developing 
and  testing  in  connection  with  new  designs  of  equipment,  thereby  making 
better  and  more  useful  equipment  designs  at  the  out-set.  Except  for  the 
invention  of  Reliability,  which  was  a  distinct  child  of  necessity,  this  is  just 
about  where  Wilks  came  in. 
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WILKS'  l  asK,  When  Wilks  toured  the  ?0  Army  installations  with  the 
Army  Mathematics  Advisory  Panel,  it  was  he  who  articulated,  "the  most 
frequently  mentioned  needs  expressed  by  the  scientific  personnel  were  for 
greater  knowledge  of  modern  statistical  theory  of  the  design  and  analyses 
of  experiments.  "  Thus,  it  is  clear  that  Wilks  recognized  at  least  a  major 
part  of  what  was  wrong  with  the  Army;  i.e.  ,  insufficient  use  of  Design  of 
Experiments  in  Research,  Development  and  Testing,  “ 

Certainly  Wilks  was  not  the  first  person  to  recognize  the  fact  that  an 
improvement  in  the  early  stages  of  the  Army  regime,  i.e.  ,  doctrine, 
tactics,  organization,  etc.  ,  has  greater  leverage  power  than  an  improvement 
in  later  stages  such  as  selection  of  equipment,  fabrication,  and  use.  The 
trend  toward  "up- stream"  improvement  began  long  before  he  appeared  on 
the  scene;  and  ranged  from  such  measures  as  advocacy  of  industrial  pre¬ 
paredness,  as  an  important  measure  towards  preserving  the  peace,  to 
various  stratagems  for  introducing  sophistication  in  the  upper  stages  of 
the  Army's  evolutionary  process.  Many  persons  deplored  the  fact  that 
traditionally  we  had  been  forced  to  begin  wars  with  the  weapons  left  over 
from  the  previous  war.  Army  Ordnance  began  to  take  measures  against 
this  ill  shortly  after  World  War  I,  and  the  then  infant  Army  Ordnance  Asso¬ 
ciation  (now  the  American  Ordnance  Association)  lent  a  patriotic  and  helping 
hand,  pursuant  to  its  slogan  advocating  industrial  preparedness  as  an  insurance 
against  war;  i.  e.  ,  a  large  production  capacity  should  exist  to  meet  a  war 
demand  for  munitions  of  the  latest  designs.  Army  Ordnance  realized  that 
it  must  have  an  eye  to  the  future  and  an  ear  to  the  ground  regarding  the  plans 
and  needs  of  the  combat  soldier,  and  therefore  sent  selected  Ordnance  Officers 
to  the  Army  Schools  ranging  from  the  Command  and  General  Staff  College 
to  the  National  War  College  to  give  them  a  close  understanding  of  the  combat 
soldier.  Liaison  officers  from  the  combat  arms  were  assigned  to  Aberdeen 
Proving  Ground,  Maryland,  to  assist  in  the  realization  of  combat  viewpoints, 
and  in  the  development  tests  of  materiel.  Shortly  after  World  War  II,  a 
number  of  percons,  including  some  Ordnance,  advocated  the  establishment 
of  a  scientific  staff  at  Headquarters,  Army  Field  Forces,  Fort  Monroe, 
Virginia,  to  assist  in  analyzing  Army  needs  and  in  stating  needs  for  new  mate¬ 
riel  In  valid  form.  Such  a  group  was  partially  formed  and  existed  for  a  year 


*The  Army  was  rot  new  to  WilkB,  In  1948  he  was  awarded  a  Joint  Army- 
Navy  Certificate  of  Merit  for  his  war-time  contributions  to  anti-submarine 
warfare  and  the  solution  of  convoy  problems. 
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or  two4.  However,  it  was  Wilks  who  undertook  systematically  the  task  of 
greatly  accelerating  the  spread  of  powerful  and  useful  statistical  techniques 
to  the  upper  echelons  of  the  Army  regime,  where  the  improvements  that 
they  enhanced  would  have  the  greatest  leverage  power. 

Even  if  Wilks  recognized  the  full  nature  of  the  job  that  he  was  doing, 
certainly,  he  did  not  have  opportunity  to  finish  the  job.  Much  remains  to  be 
done.  The  real  point  in  this  discourse  is  the  breadth  and  extent  of  the  pro¬ 
gress  made  in  the  nine  years  of  Wilks'  kindly  and  sympathetic  leadership, 
effective  persuasion,  and  hie  engendering  of  mutual  cooperation  and  helpful¬ 
ness  between  men  of  competence  with  whom  he  dealt.  Let  us  try  to  note  the 
progress,  before  any  attempt  to  assese  the  remaining  task. 

ASSESSING  THE  PROGRESS.  I  hope  that  by  the  foregoing  discussion  I 
have  led  no  one  to  believe  that  I  have  an  objective  method  of  measuring  the 
progress  of  use  of  statistical  methods  in  the  Army  during  the  1955-63  period. 

I  might  say  that  the  measuring  of  progress  in  a  field  of  science  or  engineering 
is  perhaps  one  degree  more  difficult  than  measuring  the  quantity  and  quality 
of  output  of  research  by  laboratory;  and  whereas  many  have  tried  to  do  this, 

I  know  of  no  one  who  has  really  succeeded.  The  cold  statistical  facts  are 
briefly  these; 

All  the  design  of  experiments  conferences  were  for  three  days  each, 
held  in  October  or  November,  and  conducted  at  a  number  of  Army  RfcD 
establishments. 

The  number  of  registrants  or  conferees  was  always  of  the  order  of  200. 
Attendance  was  by  invitation  and  the  number  of  invitations  was  undoubtedly 
conditioned  by  the  available  accommodations. 

The  number  of  papers  presented  at  each  conference  was  of  the  order  of 
30.  This  appear e  to  be  about  the  number  of  papers  that  can  be  presented  in 
a  three -day  conference. 

All  conferences  were  of  a  three-part  character;  Invited  papers  by 
distinguished  Statisticians,  technical  sessions  in  which  there  were  discussions 
of  recent  accomplished  work,  and  clinical  sessions  in  which  work  in  progress 
was  discussed  from  the  viewpoint  of  inviting  advice  and  criticism. 


4Later,  a  permanent  group  was  formed. 
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It  thus  appears  that  based  on  documental  evidence  the  progress  of  the 
conferences  can  be  judged  only  by  the  kinds  of  scientific  and  technical 
fields  covered  by  the  papers  and  by  the  inherent  quality  of  the  papers. 

CHARACTER  OF  PAPERS  PRESENTED.  By  and  large,  the  place  at 
which  the  conference  was  held  had  a  strong  influence  on  the  character  of 
the  papers  presented.  This  is  undoubtedly  due  tothe  fact  that  the  program 
committee  gave  some  degree  of  precedence  to  the  host  institution,  e.g., 
more  papers  bearing  on  the  field  of  medicine  were  presented  at  the  Eighth 
conference  held  at  Walter  Reed  Medical  Center  than  at  other  conferences. 
However,  in  the  statistical  fields  there  was  a  constantly  increasing  emphasis 
over  the  the  nine  years  on  the  more  sophisticated  phases  of  design  of  experi- 
ments,  screening  theory,  simulation  stratagems,  reliability,  and  techniques 
for  evaluation  of  experiments.  It  is  thus  apparent  that  expertise  on  the  part 
of  the  participants  increased  and  also  evident  that  the  use  of  statistical 
experts  in  various  fields  of  Army  activities  wae  increased  both  in  number 
of  experts  and  in  variety  of  fields  of  activity. 

Whereas,  at  the  beginning  of  the  conferences  papers  centered  largely 
around  items  of  Ordnance  materiel,  ae  the  conferences  proceeded  the  sub¬ 
ject  matter  of  the  conferences  expanded  to  include  more  emphasis  on 
systems  analysis.  Similarly,  with  the  penetration  of  statistical  methods 
into  new  fields  of  activity,  more  papers  were  devoted  to  other  than  Ordnance 
equipment.  With  the  broader  use  of  statistical  designs,  papers  appeared 
on  the  relation  of  equipment  to  organization,  and  to  new  theoretical  develop¬ 
ment  s  having  immediate  application  in  Army  use. 

A  further  change  in  the  character  of  the  papers  is  the  noticeable  effect 
of  learning  to  do  by  doing.  It  is  apparent  that  whereas  designed  experiments 
gave  greatly  Improved  results,  the  same  experiments  also  showed  defi¬ 
ciencies  in  understanding  what  one's  work  was  really  about.  For  example, 
biases  in  results  could  be  detected  that  were  readily  attributable  to  repeated 
use  of  the  same  personnel  over  the  same  terrain.  Command  exercises  had 
to  be  altered  and  new  stratagems  employed  (such  as  randomization  techniques) 
to  screen  out  the  biases  which  passed  unnoticed  when  experiments  were  of 
less  sophisticated  character.  In  fact  it  wae  precisely  the  acquirement  of 
such  evidence  that  convinced  even  non-statisticians  that  there  was  need  for 
more  movement  "up-stream".  This  wae  a  very  fortunate  circumstance 
because  it  drew  military  commanders  into  participation  in  the  planning 
of  the  experiments  and  resulted  in  a  constant  movement  of  the  sphere  of 
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activity  of  statisticians  i.-.tr  'Vi.*  Hnmain  r>f  p* r “on •  who  were  concerned  with 
policy,  tactics,  and  dec -vine.  Thus,  non-statisticians  saw  the  gains  made 
through  experiments  in  which  they,  themselves  participated. 

It  is  quite  one  thing  to  make  a  presentation  on  the  efficacy  of  a  technique, 
and  quite  another  thing  to  convince  the  hearer  that  the  use  of  the  technique 
is  important  to  his  job.  Successful  experiments  in  which  one  himself  has 
participated  (although  a  step-wise  process)  are  an  effective  method  con¬ 
vincing  one  of  the  value  of  the  methods  used.  By  way  of  contrast,  I  believe 
that  it  would  be  quite  impossible  to  suddenly  inject  into  the  military  serv¬ 
ice  (or  into  any  other  organizational  sphere,  for  that  matter)  the  concept 
and  attitude  which  is  expressed  by  the  following  quotation  taken  from  a 
Combat  Developments  Experimentation  Center  (CDCEC)  pamphlet; 

"The  ability  of  the  Army  to  carry  out  its  goals  in  the 
future  depends  upon  the  success  it  has  in  achieving  its 
combat  developments  goals  today  ...  of  developing  future 
concepts,  doctrine,  tactics,  and  techniques,  and  providing 
requirements  for  weapons,  equipment,  and  appropriate 
organisations.  " 

It  is  indeed  heartening  to  read  such  a  quotation.  This  Experimentation 
Center  has  an  area  of  over  a  quarter  of  a  million  acres,  a  brigade  of  troops, 
a  contract  with  Stanford  Research  Institute  for  Statistical  Support,  a  variety 
of  sophisticated  equipment,  including  facilities  for  computer  simulation  of 
field  experiments.  Nevertheless,  we  know  well  that  the  tasks  expressed 
in  the  quotation  are  only  beginning  and  that  only  the  first  fruits  have  yet  been 
achieved.  From  the  foregoing  example  of  CDCEC  we  can  infer  (a)  that  the 
advance  of  Statistical  Methods  in  the  Army,  during  the  past  nine  years  have 
been  great,  and  (b)  that  the  remaining  part  of  the  task,  1.  e.  ,  achieving  the 
full  nature  of  the  job  that  Wilks  undertook  is  still  a  large  one. 

WILKS’  METHODOLOGY.  If  we  hope  to  carry  on  in  substantial  meas¬ 
ure  the  task  that  lies  ahead  we  should  take  a  good  look  at  Wilks’  methods. 
Wilks  was  a  scientist  fir  the  sake  of  science,  but  he  was  aloo  a  realist  and 
wished  to  see  the  practical  results  of  applied  science  come  to  full  fruition. 


in 
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This  is  a  rare  combination  of  qualities.  *  Despite  his  many  high  scientific 
achievements  and  the  respect  in  which  he  was  held  by  his  colleagues,  he 
never  assumed  an  authoritative  position.  On  no  occasion  did  he  attempt 
to  do  a  whole  job  himself  to  the  exclusion  of  others.  On  the  contrary,  he 
always  invited  the  cooperation  of  every  person  who  could  contribute  sub¬ 
stantially  to  getting  the  job  done.  He  could  organize  and  delegate  without 
being  obvious  about  it.  In  this  way  he  secured  the  enthusiastic  support  of 
the  men  around  him.  If  anything,  he  was  more  the  servant  of  others  than 
one  demanding  services.  He  had  confidence  in  himself,  but  he  also  inepired 
confidence  in  others  that  led  them  to  venture  to  cooperate,  to  work  with  him 
and  to  work  together;  and  the  work  became  an  interesting  enterprise  to  the 
point  of  preoccupation.  In  closing,  I  would  like  to  give  a  brief  example  of 
how  the  spirit  of  Sam  Wilks  worked  towards  getting  things  done  whether  they 
were  large  or  small. 

AN  EXAMPLE  OF  W11-.KS1  WORK.  About  a  year  and  a  half  ago,  a  gentle¬ 
man  in  Georgia,  a  former  member  of  the  war-time  team  at  The  Franklin 
Institute,  who  is  intensely  interested  in  small  arms  fire  asked  several 
statisticians  including  Wilks  some  questions  about  the  inter-relations  of 
various  measurements  of  central  tendency  and  dispersion  of  shots  on  small 
arms  targets,  although  he  did  not  express  it  in  these  terms.  In  order  to 
answer  his  questions,  one  needed  to  know  the  probability  density  distribu¬ 
tions  of  several  statistical  measures  whose  distributions  were  unknown. 

These  questions  set  off  a  kind  of  chain  reaction.  It  was  possible  that  answers 
to  the  small  arms  problem  could  well  be  answers  to  othsr,  and  probably 
more  important,  problems.  Scientific  men  of  good  will,  infused  by  the 
spirit  of  cooperation  and  scientific  inquiry  contributed  what  they  knew  to  the 
general  problem;  but  it  became  evident  that  a  complete  answer  could  be 
achieved  only  by  some  research  that  would  add  a  modicum  oi  knowledge  to 
our  existing  store.  Perhaps  the  most  important  contributions  came  (later) 
from  Wilks,  Grubbs,  and  one  or  i'wo  other  colleagues  in  connection  with  their 
work  on  the  analysis  of  tracking  data  on  firings  of  long  range  missiles  at  the 


’■‘in  writing  for  the  Journal  of  the  Iloyal  Statistical  Society,  July,  1964, 
the  noted  British  Statistician,  £.  S.  Pearson  says,  .  .  it  is  hard  to 
think  of  any  mathematical  statistician  of  the  past  30  years  who  combined 
to  a  greater  extent  an  excellence  In  the  field  of  theory  with  a  power  of 
inspiring  confidence  in  government  agencies,  national  research  institu¬ 
tions,  and  educational  authorities,  as  a  wise  counseller  in  practical 
affairs.  " 
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Atlantic  Missile  Range,  The  work  turned  out  to  be  so  important  that  it  hat 
been  carefully  written  up  by  Grubbs  in  a  forthcoming  monograph.  This 
illustrates  the  humbleness,  the  spirit  and  the  methods  of  Wilks.  First,  he 
was  willing  to  lend  his  powers  to  anything  that  appeared  to  be  a  valid 
scientific  enterprise;  second,  he  had  a  keen  perception  of  what  is  fundamen¬ 
tally  important  even  though  the  context  in  which  it  was  presented  made  it 
appear  somewhat  of  casual  interest  if  not  unimportant;  third,  he  could 
engender  the  spirit  of  true  scientific  inquiry  into  his  colleagues;  fourth,  he 
could  bring  a  matter  to  a  crux  so  as  to  make  it  a  permanent  addition  to  the 
useful  knowledge  of  mankind. 

THE  WILKS'  AWARD.  It  is  important  that  the  spirit  of  Sam  Wilks  be 
carried  on,  both  for  an  unselfish  reason  and  a  selfish  reason.  Our  first 
reason  is  that  of  honoring  his  memory  in  gratitude  for  uh at  he  had  done 
for  us.  The  second  and  selfish  reason  is  that  carrying  on  the  spirit  of  his 
work  will  contribute  much  to  advancing  the  solutions  for  the  great  task  that 
he  loved  and  to  which  he  devoted  himself.  We  shall  never  achieve  the  task 
in  full;  but  each  solution  or  partial  solution  will  contribute  to  the  improve¬ 
ment  of  the  military  posture  and  safety  of  our  Country.  I  am  sure  that  Sam 
would  approve  this  second  motive.  Through  the  generosity  of  Mr.  Philip  G. 
Rust  of  Thomasvilie,  Georgia,  and  the  good  offices  of  the  American  Statis¬ 
tical  Association,  it  appears  that  a  means  has  been  found  of  achieving,  at 
least  in  part,  both  of  the  above  purposes.  An  award  will  be  created  which 
by  its  character  will  help  to  carry  on  tfca  stimulus  of  Wilks  to  Army 
Statistics. 


REFERENCES 

1.  BRL  Report  No.  897,  Proceedings  of  the  First  Symposium  on  Statistical 
Methods;  Sampling  Techniques. 

2.  Summary  Report  on  the  Establishment  of  the  Army  Mathematics  Center. 

14  October  1955.  Office  of  Ordnance  Research  (now  Army  Research 
Office  -  Durham). 

3.  A.  A.  Bennett,  On  the  Accuracy  of  Sampling.  6  January  1918.  Unpublished 
paper  in  the  files  of  the  Ballistic  Research  Laboratories. 

4.  Fowler,  Gallip,  Lock,  and  Richmond,  Aerodynamics  of  a  Spinning  Shell, 
Philosophical  Transactions  of  the  Royal  Society  of  London,  1920. 


1? 


Design  of  Experiments 

5.  F.  R,  Moulton,  New  Methods  in  Exterior  Ballistics.  University  of 
Chicago  Press,  1920. 

6.  W.  A.  Shewh&rt,  Economic  Control  of  Quality  of  Manufactured  Product. 
Van  Nostrand,  1931. 

7.  The  Proposed  System  of  Surveillance  of  War  Reserve  Ammunition, 
Zornig  and  Simon,  BRD  Report  No.  115,  1938. 

8.  Dodge,  Harold  F.  and  Harry  G.  Romig,  "Sampling  Inspection  Tables  -- 
Single  and  Double  Sampling",  John  Wiley  and  Sons,  New  York  1944; 

2nd  edition,  1959. 

9.  Abraham  Wald,  Sequential  Analysis.  John  Wiley  and  Sons,  1947. 

10.  L.  E.  Simon,  The  Relation  of  Engineering  to  Very  High  Reliability. 

Proceedings  of  the  Tenth  National  Symposium  on  Reliability  and  Quality 
Control.  January  1984.  IEEE.  i 

11.  Games,  Gods  andYlambling,  F,  N.  David\l962,  Hafner  Publishing 

Company,  New  Yoik.  \ 


\ 

\ 

V 


THE  WILKS  AWARD 


Introduction  of  Mr.  Donald  C.  Riley  by  Major  General  Leslie  E,  Simon 

Mr.  Chairman,  Fellow  Conferees,  Ladies  and  Gentlemen,  what  the  next 
two  speakers  have  to  say  is  so  closely  associated  with  my  discourse  on  Wilks 
that  I  have  been  designated  to  introduce  them. 

Ae  I  implied  at  the  end  of  my  talk,  the  establishment  of  the  Wilks  Award 
was  a  tri-partite  undertaking:  And  im^olved  the  Army  as  principal  benefi* 
ciary,  The  American  Statistical  Association  as  the  bearer  of  the  burden  of 
administration,  and  Mr.  Philip  G.  Rust  who  endowed  the  award.  Secretary 
Hawkins  has  personally  expressed  to  the  ASA  his  gratitude  for  its  competent 
and  patridtic  services. 

Mr.  Donald  C.  Riley,  Secretary-Treasurer  and  Executive  Director  of 
the  American  Statistical  Association  has  rendered  invaluable  assistance  in 
getting  swift  solutions  to  procedural  problems  he  has  been  so  kind  as  to  agree 
to  indicate  to  you  the  duties  and  obligations  of  the  ASA  in  carrying  out  the 
Wilks  Award;  and  he  will  also  announce  the  recipient  of  the  initial  Wilks 
award.  Don  Riley! 


INITIAL  WILKS  AWARD  PRESENTED  TO  DR.  FRANK  E.  GRUBBS 

Donald  C,  Riley,  Executive  Director, 

American  Statistical  Association 

Many  members  of  the  American  Statistical  Association,  as  well  as  I,  are 
glad  to  be  present  at  this,  the  Tenth  Annual  Design  of  Experiments  Conference. 
This  is  a  very  special  occasion  and  the  American  Statistical  Association  it 
glad  to  participate  during  a  uniquely  auspicious  time  in  its  long  history.  This 
year  is  the  125th  Anniversary  of  the  establishment  of  the  American  Statistical 
Association  which  recent  research  at  Stanford  has  found  to  be  the  second  oldest 
national  professional  society  inthe  United  States. 

The  American  Statistical  Association  has  always  worked  closely,  although 
usually  quite  informally,  with  agencies  of  the  Federal  Government.  For 
example,  during  the  year  it  was  founded,  1839 .  it  began  to  press  for  the 
improvement  of  decennial  censuses  and  its  representatives  played  a  major 
part  in  the  design  of  four  of  the  six  census  schedules  for  the  1850  Census.  As 
statistics  and  statistical  methodology  proliferated  vastly  since  that  time, 
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almost  all  areas  of  research  have  felt  their  impact.  Certainly  the  whole 
area  of  design  of  experiments  has  had  the  closest  association  with  statistics. 

The  annual  Design  of  Experiments  Conference  has  become  an  institution. 

General  Simon  has  reminded  you  of  the  close  association  of  Professor 
Samuel  S.  Wilks  with  this  Conference.  Most  of  you  know  that  relationship 
by  heart,  Sam  lent  his  aid  readily,  unstintingly  and  effectively  in  many 
areas.  This  was  part  of  the  genius  of  the  man. 

I  should  note  also  that  Wilks  was  the  President  of  the  American 
Statistical  Association  in  1950  and  that  he  had  always  done  much  for  the 
Association,  He  also  helped  to  carry  on  in  another  area  the  close  relation 
between  the  Association  and  the  Federal  Government.  Just  the  day  before 
he  died  he  participated  as  a  member  of  the  Advisory  Committee  on 
Statistical  Policy  to  the  Office  of  Statistical  Standards  in  the  Bureau  of 
the  Budget.  The  Office  of  Statistical  Standards  requires  consultation  from 
time  to  time  at  a  high  level  in  its  work  as  the  central  statistical  coordinating 
body  of  the  Federal  Government.  This  Advisory  Committee  consists  largely 
of  former  ASA  Presidents  and  Wilks  was  one  of  its  "founding  fathers.  " 

As  mentioned  in  General  Simon's  address,  the  ASA  has  recently  had 
the  opportunity  to  be  of  further  eervice,  By  joint  agreement  between 
representatives  of  the  Army,  Mr.  Philip  G.  Rust  and  the  ASA,  the  Samuel 
S.  Wilks  Award  has  been  established.  The  Award  will  consist  of  a  medal 
and  sui  honorarium.  The  ASA  has  accepted  the  obligation  of  administering 
the  Award  in  accordance  with  guidance  and  criteria  which  are  consonant 
with  law  and  with  the  wishes  of  Army  representatives,  Mr.  Rust  and  the  ASA. 

Annually,  ASA  has  agreed  that  an  appropriate  committee  be  selected 
(or  appointed)  to  select  the  awardee,  based  on  the  criterion  that  he  is  a  per¬ 
son  whom  the  committee  regards  as  deserving  of  the  award,  based  primarily 
on  his  contribution  (either  recent  or  past)  to  the  advancement  pf  scientific 
or  technical  knowledge,  ingenious  application  of  existing  knowledge,  or 
successful  activity  in  the  fostering  of  cooperative  scientific  efforts  which 
have  only  coincidentally  benefited  the  Army.  The  award  shall  be  made  with 
the  intent  of  recognising  the  personal  and  intellectual  accomplishments  of  the 
individual  and  shall  not  be  given  with  the  intent  of  supplementing  the  individual's 
salary,  providing  him  with  compensation,  or  advancing  the  interests  of  the 
donor  or  trustee  of  the  endowment. 

The  American  Statistical  Association  has  been  asked  to  invest  the  funds 
so  generously  turned  over  to  it  for  this  purpose  and  I  am  sure  that  its  Board 
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of  Directors,  which  has  given  its  wholehearted  approval,  feels  honored  in 
being  asked  to  join  in  honoring  Sam  Wilks.  ASA  will  need  to  consult  very 
closely  with  those  of  you  who  have  helped  to  develop  the  annual  Design  of 
Experiments  Conferences,  in  the  selection  of  an  Annual  Sam  Wilks  Award 
Committee.  I  believe  that  Dr.  Albert  H.  Bowker,  the  President  of  the 
American  Statistical  Association  this  year,  will  be  able  to  announce  this 
Committee  shortly. 

As  executive  Director  of  the  ASA,  I  have  the  honor  to  announce  that 
Dr.  Frank  E.  Grubbs  of  the  Army's  Ballistic  Research  Laboratories  has 
been  selected  to  receive  the  "initial,  "  not  the  first,  Samuel  S.  Wilks  Award. 
As  is  not  unusual  in  the  initial  award  of  an  honor,  Dr.  Grubbs  was  selected 
not  by  the  process  governing  the  first  and  subsequent  recipients,  but  rather 
by  unanimous  agreement  of  those  concerned  with  the  establishment  of  the 
Award.  He  is  so  selected  because  of  his  close  working  relationship  with 
Wilks,  and  especially  because  of  his  contributions  along  with  Wilks  to 
solutions  and  clarification  of  simple  measures  of  dispersion,  which  are 
deemed  useful  to  riflemen,  balHsticians ,  and  statisticians  in  general. 

I  have  no  medal  to  present  to  Dr.  Grubbs,  because  the  medal  has  not 
yet  been  struck;  but  it  will  Jbe  presented  at  the  earliest  appropriate  oppor¬ 
tunity,  after  it  is  available. 

Incidentally,  I  will  not  be  able  to  attend  the  banquet  here  tomorrow 
evening  because  I  agreed  long  ago  to  attend  the  inauguration  ceremonies  in 
New  York  of  Dr.  Bowker  as  Chancellor  of  the  combined  Universities  of 
the  City  of  New  York  which  was  organized  a  few  years  ago. 

The  American  Statistical  Association  will  want  to  continue  to  advise 
closely  with  the  Conference  and  will  be  glad  to  ask  its  auditor  to  render 
a  brief  auditing  report  each  year  if  this  seems  satisfactory  to  those  who 
have  been  so  close  to  Sam  Wilks,  General  Simon  and  especially  Mr.  Philip 
G.  Rust,  who  has  been  so  generous  and  public  spirited  in  making  the  award 
possible.  I  should  like  to  join  in  thanking  Mr.  Rust  most  profoundly. 

INTRODUCTION  OF  MR.  PHILIP  G.  RUST 
BY  MAJOR  GENERAL  LESLIE  E.  SIMON 

Mr.  Chairman,  Fellow  Conferees  and  Ladies  and  Gentlemen. 
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We  now  come  to  the  third  and  last  speaker  ir.  this  phase  of  our  honoring 
Sam  Wilks,  Mr.  Philip  G.  Rust  of  Winnstead  Plantation,  Thomasville,  Georgia. 
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Mr.  Rust  is  a  very  modest  man,  and  more  adept  at  understatement  than  a 
typical  Britisher.  It  was  only  under  pressure  personally  exerted  by 
Secretary  Hawkins  that  we  succeeded,  first,  in  overcoming  hie  insistence 
that  he  remain  anonymous,  second,  in  getting  him  to  attend  this  conference, 
and  third,  in  persuading  him  to  present  the  honorarium  to  the  initial  recipient 
of  the  Wilks  Award,  Dr.  Grubbs. 

Mr.  Rust  purports  to  be  practically  innocent  of  theoretical  and  applied 
statistics;  but  if  under  pressure,  he  can  cite  statistical  literature  by  page 
and  paragraph  showing  each  historical  advance  in  statistical  measures  of 
dispersion;  he  professes  no  close  association  with  science  and  engineering, 
but  1  find  that  he  was  not  only  a  research  chemist  for  over  ten  years,  but 
also  returned  to  science  and  engineering  during  World  War  II;  he  lays  claim 
only  to  being  a  Georgia  farmer,  but  he  has  contributed  to  ASA  the  funds 
necessary  to  establish  the  award  commemorating  his  old  friend,  Sam  Wilks, 
contributing  to  the  welfare  of  the  military  services,  and  fostering  science 
in  general. 

With  these  cautionary  remarks,  I  deem  it  a  privilege  and  an  honor  to 
introduce  Mr.  Philip  G.  Rust. 

THE  CONCEPTION  OF  THE  WILKS  AWARD 
Philip  G.  Rust 

Winnstead  Plantation,  Thomaeville.  Georgia 

Mr.  Chairman  and  members  of  the  audience  you  have  heard  a  most 
informative  talk  by  General  Simon  on  "The  Stimulus  of  S.  S.  Wilks  to  Army 
Statistics.  "  Then,  on  Thursday  we  may  look  forward  to  Dr.  Eisenhart's 
"Sam  Wilks  as  I  Remember  Him.  " 

In  view  of  the  newly  established  Association's  Wilks  Award,  concisely 
described  to  you  by  Mr.  Donald  C.  Riley,  the  Executive  Director  of  the 
American  Statistical  Association;  it  is  appropriate  that  I  briefly  discuss  the 
conception  of  this  award. 

Back  in  the  dark  days  of  1944,  Dr.  Wilks  and  I  were  headed  north  from 
Washington,  by  train;  he  to  Princeton,  and  I  to  my  home  in  Wilmington. 

At  the  time,  I  was  at  The  Franklin  Institute,  working  on  .  50  calibre  barrel 
erosion,  and  also  as  the  un-official  translator  of  pertinent  technical  works, 
in  passing,  I  would  state  that  the  Institute  work  was  less  statistical  than  of 
the  ear  drum  rupturing  variety. 
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On  this  train  trip,  I  happened  to  mention,  that  for  my  spars  time 

had  beer,  devoted  io  certain  statistical  measure*  of  shots  on  a  target.  After 
telling  Dr.  Wilks  about  the  firing  of  hundreds  of  .  22  calibre  targets,  from 
rest;  to  get  an  empirical  measure  of  the  distribution  of  "extreme  spread", 
he  asked  if  I  had  started  any  theoretical  work  on  the  subject.  (Incidentially, 

"  extreme  spread"  is  defined  as  the  separation  distance  of  the  two  widest 
apart  shots.)  His  interest  increased  when  it  was  mentioned  that  I  had  made 
a  start  by  generating  a  few  hundred  artificial  targets  by  using  pairs  of  random 
numbers  in  the  well-known  bi-variate  circular  distribution.  Equal  likelihood 
of  angular  distribution  was  assumed,  with  no  systematic  errors. 

The  shots  were  laboriously  plotted  on  cross  section  paper,  and  ihe 
extreme  spread  and  other  parameters  examined.  It  is  of  interest  to  note 
that  the  fired  targets  and  the  plotted  ones  are  extremely  close. 

About  this  time,  my  travelling  companion  suggested  that  he  disembark 
at  Wilmington,  also.  I  had  the  feeling  that  he  wanted  to  explore  the  applica¬ 
tion  of  these  data  to  other,  more  vital  matters.  He  stated  that  he  had  an 
exceptional  graduate  student  who  might  be  given  the  job  of  finding  the  true 
distribution  of  "extreme  spread". 

Eight  or  ten  years  went  by,  and  our  contacts  were  largely  by  phone.  He 
assured  me  that  he  was  still  interested,  and  working  on  target  problems; 
but  that  as  yet,  this  distribution  had  not  been  discovered.  The  possibility 
of  Monte  Carlo  methods  on  a  to-be-aquired  computer  were  discussed.  Then 
on  10  August  1963,  I  received  a  long-hand  letter  saying  that  a  7090  computer 
was  at  hand,  busily  working  on  related  matters. 

While  waiting  for  promised  data  from  Dr.  Wilks,  1  approached  General 
Simon  about  the  subject.  He  later  discussed  it  with  Dr.  Frank  Grubbs  of 
Aberdeen,  who  subsequently  brought  forth  an  extremely  useful  manuscript, 
soon  to  be  published. 

Finally,  on  Dr.  Wilk's  1963  Christmas  card,  he  stated  that  the  target 
problem  was  tied  in  with  tracking  work  on  the  Atlantic  Missile  Range. 

General  Simon;  with  his  very  orderly  mind,  and  sense  of  the  fitting, 
then  suggested  the  idea  of  the  annual  A.  S,  A.  Wilke  Award.  This  idea  was 
greeted  enthusiastically  by  all  concerned. 
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What,  then  could  be  more  fitting,  than  that  Dr.  Frank  E.  Grubht  should 
be  the  recipient  of  the  initial  award. 

And  now,  it  gives  me  great  pleasure  to  hand  Dr.  Grubbs  the  initial 
honorarium  and  the  assurance  of  its  accompanying  medal  on  ito  completion, 
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DEVELOPMENT  OF  THE  DESIGN  OF  EXPERIMENTS 
OVER  THE  PAST  TEN  YEARS* 

Oscar  Kempthorne 
Iowa  State  University,  Ames,  Iowa 

INTRODUCTION.  The  main  aspects  of  experimentation  on  which  pro¬ 
gress  has  been  made  in  the  past  10  years  appear  to  be  the  following: 

(a)  the  analysis  of  experiments 

(b)  the  development  of  incomplete  block  designs 
and  (c)  the  investigation  of  multifactorial  systems. 

I  shall  have  just  a  few  words  to  say  about  the  first  two  items  and  shall 
spend  practically  all  my  time  on  the  third  item. 

THE  ANALYSIS  OF  EXPERIMENTS.  In  the  last  15  years  or  so,  statis¬ 
ticians  have  become  concerned  about  the  assumptions  that  are  commonly 
made  in  the  analysis  of  comparative  experiments.  The  common  analysis 
is  to  use  the  matrix  model 


y  »  Xp  +  e 

in  which  y  is  the  vector  of  observations,  X  is  a  matrix  of  known  elements, 

P  is  a  vector  of  unknown  parameters,  and  the  vector  e  of  errors  is  assumed 
to  consist  of  components  which  are  normally  and  independently  distributed 
around  aero  with  constant  variance.  The  obvious  questions  aboxit  such  a  model 
are: 


(1)  why  use  y,  and  why  not  a  defined  function  of  y,  such  as  log  y 
or  l/y,  or  any  of  a  host  of  other  possibilities  ? 

(2)  is  the  model  linear  in  the  parameters,  that  is,  is  the  expectation 
Xp  ,  correct  ? 

(3)  is  the  assumption  about  the  errors  correct? 

In  recent  years  there  has  been  considerable  attention  to  these  questions, 
primarily  by  Anscombe  (1961),  Tukey  (1962)  and  Anscombe  and  Tukey  (1963), 
the  work  dating  back  to  Tukey's  one  degree  of  freedom  for  non -additivity. 
This  has  led  to  the  topic  -  residual  analysis  -  which  is  now  an  every  day 
phrase. 

"■Prepared  in  connection  with  work  on  Contract  AF  33(615)-1737, 

Office  of  Aerospace  Research,  United  States  Air  Force, 

Wright-Patterson  Air  Force  Base,  Ohio. 
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Mattes  uiir claied  to  residual  analysis  but  part  of  data  analysis,  are 
topics  such  as  the  question  of  multiple  comparisons,  the  effects  of  preliminary 
test  on  conclusions,  random,  mixed  and  fixed  models,  and  randomisation 
theory  of  experimental  inference.  1  shall  not  discuss  these. 

THE  DEVELOPMENT  ON  INCOMPLETE  BLOCK  DESIGNS.  Incomplete 
block  designs  were  developed  to  control  variability  among  the  experimental 
units.  The  original  incomplete  block  designs  were  given  by  Yates  in  the  30's, 
and  in  1939  Bose  and  Nair  developed  a  fairly  general  class  of  such  design. 

Since  that  time  there  has  been  a  development  of  blocking  theory  with  regard 
to 

(a)  The  structure  and  existence  of  incomplete  block  plana 

(b)  the  arrangement  of  factorial  designs  in  incomplete  block  design*, 

Such  development  is  very  desirable,  but  it  i*  agreed  by  moat,  I  imagine, 
that  the  impact  of  this  work  on  the  conduct  of  experiments  is  not  great. 

Roughly  speaking  we  have  had  for  many  year*  an  array  of  ineomplate  block 
designs  which  provides  an  adequate  basis  for  choice  for  most  experimental 
situations. 


THE  INVESTIGATION  OF  QUALITATIVE  FACTORIAL  SITUATIONS.  It 
is  essential  to  differentiate  between  multifactorial  situations  in  which  the 
factors  are  qualitative  and  in  which  the  factor*  are  continuous  or  quantitative. 
In  the  former  case  the  structure  of  the  totality  of  possible  information  consist* 
of  the  true  yields  and  variability  for  each  of  the  possible  factor  combinations, 
In  the  latter  case  the  totality  of  possible  information  is  a  functional  relation¬ 
ship  of  yield  to  the  levels  of  the  factor*  or  variables.  So  in  the  qualitative 


case ,  if  one  has  factors  say  a,  b,  q.  .  .  ,  with  levels  denoted  by  a^,  b^ ,  c^, ,  .  .  , 
the  underlying  formula  for  yield  will  be  of  the  form  y(a^,  b^,  c^,  .  .  .  , )  - 
f(i,j,k,  , .  .  ,)  +  error  where  the  function  is  defined  only  for  the  factor  levels 


i,j,k,...,  in  the  situation.  In  other  words,  the  model  has  to  be  a  classi- 


ficatory  model.  Classificatory  models  can  be  linear  as  exemplified  by 


yijk  =  ■*  +  Qi  +  Pj  +  (aPJjj  +  Vk  +  etc  +  error, 


or  can  be  non-linear,  as  for  example 
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y 


ijk 


Q, 

1 


V^k 


+  error. 


Essentially  no  theory  exists  for  non-linear  classificatory  models,  and  I  am 
of  the  opinion  that  this  is  a  real  gap  in  our  knowledge. 

In  the  case  of  study  of  the  full  set  of  factorial  combinations,  one  of  the 
basic  problems  is  error  control  and  systems  of  confounding  were  developed 
for  symmetrical  systems  in  the  30' s.  There  have  been  a  few  developments 
in  recent  years  with  regard  to  confounding  for  the  asymmetrical  case,  and 
also  some  clarification  of  the  mathematical  structure  of  factorial  experi¬ 
ments  [e.  g.  Kurkjian  and  Zelen  (1962)]  .  I  imagine,  however,  that  examina¬ 
tion  of  the  full  set  of  factorial  combinations  is  rarely  appropriate  except 
possibly 

(a)  when  most  of  the  factors  have  2-levels,  with  perhaps  two  three  - 
level  factors, 

and  (b)  in  the  case  of  experiments,  like  in  agronomy,  for  which  there  is 
a  long  essentially  unalterable  interval  of  time  from  executing  the 
design  to  obtaining  the  experimental  results,  on  the  basis  of  which 
to  plan  another  experiment. 

There  has  been  one  development  of  analysis  which  seems  to  be  very 
informative,  when  the  totality  of  treatment  degrees  of  freedom  can  be 
partitioned  into  meaningful  orthogonal  single  degrees  of  freedom,  the  half- 
normal  plot  of  Daniel  (1959).  The  idea  of  half-normal  plotting  is  the  very 
elementary  one  of  lookifig  at  the  distribution  of  the  totality  of  single  degree 
of  freedom  contrasts,  and  to  observe  which  ones  are  outliers.  The  half¬ 
normal  plot  is  a  convenient  way  of  doing  this.  In  general  tight  rules  of 
significance  for  examining  the  realized  half-normal  plot  do  not  exist.  The 
procedures  of  half-normal  plotting  have  been  generalized  to  the  case  of  a 
multivariate  response  by  Wilk  and  Gnanadesikan  (1963,  1964). 

In  the  case  of  the  linear  classificatory  models  it  is  obvious  that  the 
simplest  design  problenQ.is  to  estimate  the  effects  under  the  assumption 
of  no  interactions.  Effective  designs  for  this  cate  have  now  been  available 
for  several  years.  The  earliest  example  of  such  a  design  was  given  by 
Tippett  and  is  described  in  Fisher's  "Design  of  Experiments"  for  the  test¬ 
ing  5  factors  in  25  trials.  In  the  1940's  the  following  sets  of  main  effect 
plana  were  developed; 
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In  all  these  cases  p  is  a  prime  number  or  a  power  of  a  prime  number. 
Tukey  (1959)  and  Addelman  (1962)  showed  that  these  symmetrical  main 
effect  plans  can  be  used  to  develop  very  reasonable  main  effect  plans  for 
asymmetrical  factorial  situations. 

In  the  1940's  Finney  (1945)  formulated  the  general  idea  of  fractional 
replication,  which  is  closely  related  to  the  idea  of  confounding.  It  is 
interesting  to  note,  in  passing  that  Fisher  was  primarily  interested  in 
systems  of  confounding,  and  it  was  not  adequately  realized  for  some  years 
that  he  had  in  fact  developed  incidentally  the  series  of  main  effect  plans 
mentioned  above.  The  idea  of  fractional  replication  is  to  use  a  subset  of 
the  totality  of  treatment  combinations  chosen  on  the  basis  of  the  definition 
of  effects  and  interactions.  Obvious  candidates  as  useful  designs  in  this 
class  are  the  main  effect  plans,  and  the  designs  which  permit  estimation 
of  all  main  effects  and  two -factor  interactions. 

Also  in  1946,  Rao  (1947)  formulated  the  idea  of  orthogonal  arrays.  An 
array  (N,k,  s,t)  is  a  collection  of  N  treatment  combinations  out  of  the 
K 

totality  s  of  treatment  combinations  possible  with  -k  factors  each  at  s 
levels,  such  that  every  combination  of  every  subset  of  t  factors  occurs 
equally  frequently.  The  value  t  is  called  the  strength  of  the  array.  An 
array  of  strength  2  is  an  orthogonal  main  effect  plan.  With  an  array  of 
strength  3,  no  main  effect  is  confounded  with  two-factor  interactions ,  but 
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some  two-factor  interactions  are  mutually  confounded.  Ar.  nr  ray  of 
strength  4  enables  the  orthogonal  estimation  of  all  main  effects  and  two- 
factor  interactions ,  and  so  on.  Clearly  the  enumeration  of  main  effect 
plans,  two -factor  interaction  plans  etc.  is  related  to  the  enumeration  of 
orthogonal  arrays.  Box  and  Hunter  (1961a,  b)  have  given  a  rather  detailed 
account  of  the  possibilities  of  fractional  replication  with  2-level  factors, 
using  the  term  degree  of  resolution  instead  of  the  strength  of  array  of  Rao. 
A  design  of  resolution  III  gives  main  effects  estimates,  which  will  be 
biassed  by  two-factor  interactions,  A  design  of  resolution  IV  gives  main 
effects  unconfounded  with  two-factor  interactions,  but  with  the  two-factor 
interactions  somewhat  interconfounded  and  a  design  of  resolution  V  is  a 
two-factor  interaction  -  clear  design.  They  show  that  a  design  of  reso¬ 
lution  III  repeated  with  reversed  signs  gives  a  design  of  resolution  IV. 

They  discucs  extensively  the  arrangement  of  fractionally  replicated  plans 
in  blocks.  They  also  examine  the  possibility  of  plans  which  estimate 
interactions  among  all  of  a  subset  of  the  factors  with  the  effacts  of  another 
subset  of  factors,  the  former,  being  regarded  ae  major  variables  and 
the  latter  as  minor  variables.  -  For  example,  they  give  a  2^"^  plan 
which  enables  the  estimatioiVof  all  effects  and  interaction!  among  4  major 
variables  and  the  main  effects  of  16  minor  variables.  Box  and  Hunter 
(1961b)  give  the  possible  two -factor  intsraction  clear  fractions  in  blocks 


for  up  to  U  factors. 

The  possibilities 

are  as 

follows: 

No,  of  factors 

5 

6 

7 

8 

9  10 

11 

No.  of  observations 

16 

32 

64 

64 

128  128 

128 

No.  of  blocks 

1 

2 

8 

4 

8  8 

8  . 

Addelman  (private  communication)  has 

found  s 

i  217*9  resolution  V  plan 

in  8  blocks  of  32.  These  plans  enable  orthogonal  estimation  of  all  ths 
offsets  and  two-factor  interactions  and  appear  to  be  the  minimal  designs 
which  allow  orthogonal  estimates. 

If  one  ia  prepared  to  relax  the  orthogonality  requirement,  one  can 
obtain  reasonably  precise  estimates  with  irregular  fractions  (Addelman, 
1961  and  Whitwell  and  Morbey,  1961).  For  instance  Addelman  gives  a 

3  7  3  8  3  9 

fraction  j  of  a  2  factorial,  of  a  2  ,  and  jg-  of  a  2  to 

estimate  all  main  effects  and  2-factor  interactions.  Whitwell  and  Morbey 
give  a  design  using  96  observations  which  allows  the  estimation  of  the 
main  effects  and  all  but  3  of  the  two -factor  interactions  of  11  factors. 
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Fractional  replication  of  the  3°  factorial  system  is  much  more 
difficult,  as  soon  as  one  wishes  to  estimate  two -factor  interactions.  In 
the  case  ot  5  factors,  for  instance,  the  smallest  plan  which  allows 
estimation  of  two-factor  interactions  is  a  1/3  replicate  requiring  81 
observations.  The  problems  of  enumerating  two-factor  interaction  cle^r 
plane  fpr  the  3n  factorial  system  appear  to  be  rather  difficult.  Bose, 

Bush,  Seiden  and  others  have  worked  on  the  enumeration  of  orthogonal 
arrays  and  on  the  maximum  number  of  factors  which  can  be  accommodated 
with  a  given  number  of  observations,  but  the  situation  is  still  quite  unclear. 
Obviously,  the  main  experimental  interest  is  in  arrays  of  strength  4, 

One  possible  way  of  examining  a  multifactor  situation  is  by  some  use  of 
random  sampling  of  the  totality  of  treatment  combinations.  This  idea  was 
first  put  forward,  it  appears,  by  Satterthwaite  (1959)  and  attempts  have 
been  made  to  develop  a  theory  of  inference  from  such  sampling,  e.  g.  by 
Dempster  (I960,  1961),  It  appears  that  the  situation  is  very  difficult. 
Ehrenfeld  and  Zacks  (1961),  Zacks  (1963)  and  Ehrenfeld  and  Zacks  (1963) 
have  examined  two  procedures  of  random  sampling  the  totality  of  treatment 
combinations  which  are  based  on  fractional  replication.  It  would  appear  that 
considerable  further  development  is  needed  of  ways  of  sampling  the  totality 
of  treatment  combinations  and  of  analysing  the  resultant  sample. 

The  general  moral  to  be  drawn,  then,  with  regard  to  multifactor 
(qualitative)  experiments,  is  that  it  is  easy  to  examine  for  main  effects, 
more  or  less  regardless  of  the  number  of  levels,  but  that  examination  for 
interactions  can  in  general  be  done  at  all  easily  only  with  two  levels  for 
each  factor.  It  is  likely  that  if  the  requirement  of  orthogonality  is  waived, 
plans  requiring  reasonable  numbers  of  observations  can  be  developed. 

THE  INVESTIGATION  OF  DEPENDENCE  OF  A  YIELD  VARIA3LE  (y)  ON 
k  CONTINUOUS  CONTROL  VARIABLES  (x^x,,  .  .  .  .xj.  It  would  seem  that 

while  there  are  many  aspects  of  the  dependence  of  &  yield  variable  on  k 
control  variables  which  can  be  varied  continuously,  one  can  "spin  off"  one 
problem  which  is  quite  different  in  nature  from  all  the  others,  and  that  is 
the  optimization  problem,  namely  to  determine  the  values  of  x^,  x^, .  .  .  ,  x^, 

such  that  the  yield  is  a  maximum  (or  minimum),  Of  course  there  are  situ¬ 
ations  in  which  there  are  several  yield  variables,  say,  y,  ,y_,..  .  ,y  and 

Id  m 

the  problem  may  be  more  complex,  such  as  to  determine  the  combination 
(x  , x  , .  .  ,  ,x  )  for  which  y  is  a  maximum,  subject  to  restraints  of  the 

lb  K  1 

type  y2  <  k2,  y3  >  k3  and  so  on. 
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ukumum  sttiiMiNU.  The  work  in  iiii*  «r*«  utterly  r.iiv; ,  cnc  -z.de ? 
at  a  time  experimentation,  until  the  work  of  Box  and  Wilaon  (1951)  to  whom 
great  credit  is  due  for  tackling  the  problem  with  some  degree  of  aophistication. 
X  shall  enumerate  briefly  the  steps  of  the  Box- Wilson  procedure.  They  are: 

(1)  local  exploration  around  a  guessed  optimum  by  means  of  a  design 
which  enables  the  fitting  of  the  relationship 

y  =  bo  +  bft  +  b2x2  +.  • .  +  Vk  1 


(2)  proceeding  along  a  line  in  the  direction  of  steepest  ascent  in  the 
units  chosen  to  an  optimum  on  that  lins; 

(3)  local  exploration  around  this  newly  obtained  optimum  as  in  (1)  ; 

(4)  proceeding  along  a  new  steepest  ascent  direction  as  in  (2); 

(5)  repetition  of  steps  (3)  and  (4); 

(6)  when  there  ceases  to  be  a  pay-off  from  this  process,  perform 
local  experimentation  around  the  achieved  sub-optimum  to  enable 
the  fitting  of  a  second  degree  dependence  of  y  on  the  x'e; 

(7)  do  a  mathematical  analysis  of  the  achieved  second  degree 
relationship.  That  is,  if  one  has  found  the  relationship 

V  •  30  +  Vixi  ' 


then  one  can  make  a  linear  transformation  of  x^,x2, 
say  *2"  ' '  ’  *k  *° 


y 


*0 


,  2 

Vi 


,  2 
*2*2 


+, 


Vk 


(8)  this  representation  enables  one  to  see  the  form  oi  the  relationship 
of  y  to  the  z's  in  the  neighborhood  of  the  sub-optimum  achieved 
earlier.  If  all  the  are  negative,  the  optimum  is  at  the  point 

where  all  the  z's  are  zero.  If  some  are  zero  there  is  a  subspace 
of  optima.  If  for  example  X1  is  zero  and  the  others  are  negative 
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the  optimum  (maximum)  is  achieved  wherever  z^  which  is  a  linear 

function  of  the  x's  is  zero.  If  of  course  any  X^  is  positive  the 
maximum  is  not  at  all  defined  by  the  fit. 

Apart  from  steps  (6),  (7)  and  (8)  this  is  the  standard  iterated  steepest  ascent. 
Obviously  the  procedure  was  developed  for  the  optimization  of  a  production 
process  in  which  only  local  experimentation  is  possible  so  as  not  to  disrupt 
production. 

The  procedure  suffers  from  the  well-known  disadvantage  of  steepest 
ascent  in  that  progress  may  be  excellent  for  the  first  few  steps  but  then 
becomes  very  slow.  Of  course  steps  (6),  (7)  and  (8)  were  inserted  by  Box 
and  Wilson  to  take  care  of  this. 

A  line  of  attack  on  this  problem,  which  is  closely  related  to  the  Box- 
Wilson  approach,  consists  of  trying  to  develop  algorithms  which  will  give 
rapid  convergence  to  the  optimum  if  the  variable  to  be  optimized  y,  say, 
is  known  without  error  and  is  of  the  form 

y  =  bg  +  b'o.  +  x'Cx 

in  which  C  is  negative  definite,  so  that  a  unique  optimum  exists.  One  then 
attempts  to  determine  the  properties  of  the  algorithm  if  the  relationship  of 
the  y  to  the  x's  is  not  of  the  postulated  form,  and  if  y  is  known  only 
with  erro.\  The  methods  I  know  of  which  have  this  structure  are  the  follow¬ 
ing,  the  method  of  parallel  tangents  due  to  Shah,  Buehler  and  myself  (1964), 
and  the  method  of  Fletcher  and  Powell  (1963).  The  method  of  Fletcher  and 
Powell  is  based  on  a  guess  of  the  matrix  C,  which  would  ordinarily  be 
taken  as  the  unit  matrix,  and  on  successive  line  searches,  the  directions  of 
which  change  on  the  basis  of  previously  determined  gradients  and  on  the  steps 
to  the  optima  on  the  lines.  The  method  of  parallel  tangents  is  really  just  an 
acceleration  of  the  initial  steps  of  the  Box-Wilson  procedure  which  removes 
the  necessity  of  fitting  a  second  order  relationship.  One  variant  of  the  method 
of  parallel  tangents  has  a  particularly  simple  structure: 
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in  which  the  lines  labelled  S.  A.  are  steepest  ascent  lines  and  the  dashed 
lines  are  acceleration  lines.  In  the  abeence  of  error  and  with  k  dimensional 
ellipsoidal  yield  contours  the  maximum  as  reached  at  the  point  labeled  2n. 

There  are  other  intuitive  methods  such  as  pattern  search  of  Hooke  and 
Jeeves  (1962),  and  methods  using  sectioning  of  the  factor  space  on  the  basis 
of  tangent  planes  to  the  yield  contours  (Wilde,  1964). 

Th>?se  methods  appear  to  use  with  some  degree  of  effectiveness,  the 
information  that  is  accumulated  by  *he  separate  local  experiments.  A  real 
difficulty  from  a  theoretical  viewpoint  is  to  evaluate  the  properties  of  all 
these  methods,  including  the  Box-Wilson  method,  in  the  presence  of  error. 

Just  how  important  it  is  from  a  practical  viewpoint  to  establish  tight 
clean  mathematical  results  about  the  performance  of  these  strategies  in  the 
presence  of  error  is,  I  believe,  a  moot  point.  It  would  of  course  be 
valuable  from  an  aesthetic  viewpoint  to  have  such  information,  but  the 
difficulties  of  obtaining  information  of  practical  value  seem  to  be  tremendous. 
It  is  clear  that  the  strategies  described  above  are  so  loosely  defined  that 
they  cannot  be  subjected  to  precise  mathematical  evaluation.  Answers  to 
such  questions  as  (a)  how  does  one  explore  locally?  (b)  what  is  the  "spread" 
of  the  local  design?  (c)  how  does  one  search  for  the  optimum  on  a  line? 

(d)  how  does  one  decide  when  to  terminate  ?,  are  not  given  by  th  procedures. 
They  are,  however,  questions  which  the  user  will  be  able  to  make  choices 
which  must,  of  course,  be  somewhat  arbitrary  but  which  will  be  modified 
as  information  accumulates.  If  the  local  experimentation  does  not. indicate 
clearly  that  there  is  a  direction  in  which  improvement  can  be  made,  more 
local  experimentation  will  be  done,  presumably  by  either  repeating  what 
was  done  before  or  by  "pulling  in"  the  local  design  and  repeating.  Also,  it 
is  obvious  that  the  experimenter  will  survey  the  totality  of  information  obtained 
up  to  any  particular  point  in  the  process  and  will  modify  the  algorithms  if  he 
can  spot  a  pattern  in  the  response  relationship. 

A  direct  attack  on  the  optimization  problem  with  error  was  made  by 
Kiefer  and  Wolfowitz  (1952)  with  work  related  to  that  of  Robbins  and  Monro 
(1951)  who  developed  a  stochastic  approximation  scheme  for  finding  the  value 
x,  at  which  the  expected  value  M(x)  of  a  random  variable  y(x)  takes  a  partic¬ 
ular  vj.lue.  The  Kiefer -Wolfowitz  procedure  is  as  follows:  for  the  case  of 
optimization  in  one  dimension  choose  two  sequences  of  positive  numbers, 

c  ,  a  ,  such  that  lim  c  =  0,  Z  a  =  «  ,  2  a  c  <  »  and  £  a^c~^  <  ® ,  as, 
n  n  n  n  n  n  n 

for  example  a  =  —  ,  c  =  A,  ;  take  an  arbitrary  z,  and  then  use 
n  n  n  1/  j  1 
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z 


n+1 


z 

n 


+ 


+ 


Then  z  converges  stochastically  to  the  point  z  at  which  E  y(z)  is  a 
maximum.  Kiefer  and  Wolfowitz  (1952)  state  that  there  remain  the  problems 
of  choices  of  sequences  a^  and  cn  which  will  be  optimal  in  some  sense, 

and  the  specification  of  a  stopping  rule.  This  line  of  work  has  been 
developed  considerably  by  Blum  (1954),  Dvoretsky  (1956),  Kesten  (1958) 
and  by  Sacks  (1958),  and  others  to  the  multidimensional  case. 


It  is  not  clear  at  all  what  the  attitude  of  the  practical  statistician  should 
be  to  these  very  different  approaches,  Kiefer  (1959)  states  that  methods 
such  as  the  Box-Wilson  one  or  the  others  of  the  same  flavor,  "cannot 
in  their  present  state  have  any  role  in  satisfactorily  solving  these  problems, 
since  they  have  no  guaranteed  probability  properties  and  are  not  even  well* 
defined  rules  of  operation.  "  Barnard,  however,  in  discussion  of  Kiefer's 
paper,  disagr  eed  and  took  the  view  that  rules  of  operation  which  are  not 
well-defined  may  be  preferable  to  the  rules  which  are.  It  would  seem  that 
the  guaranteed  property  of  convergence  with  probability  one  with  an  infinite 
number  of  observations  is  small  comfort  to  the  practical  man,  even  though 
it  was  obviously  not  easy  to  develop  procedures  for  which  one  can 
prove  the  property. 


What  we  really  lack  are  accounts  of  actual  experiences  with  the  various 
methods.  Perhaps  a  good  practical  strategy  is  to  use  the  "deterministic" 
schemes  at  first,  and  then  turn  to  the  stochastic  schemes  when  the  former 
cease  to  give  advances. 


RESPONSE  SURFACE  EXPLORATION.  I  now  turn  to  the  problem  of  studying 
the  dependence  of  a  yield  variable  y  on  continuous  control  variables 
(xj.x-j. .  .  .  ,x^)  which  has  been  termed  a  response  surface  exploration  by 

Box  and  his  co-workers. 


The  great  bulk  of  the  work  on  this  problem  has  been  by  Box  and  his 
associates,  stemming  back  to  the  famous  Box-Wilson  paper  (1951).  The 
background  for  the  work  is  the  paper  by  Box  (1952)  on  first  order  multi¬ 
factorial  designs,  which  I  have  to  review  even  though  it  was  done  more  than 
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10  years  ago.  Here  Box  specified  the  amount  of  variation  of  each  variable  or 
factor  by  defining  the  scale  unit  S.  for  the  i-th  variable  as 


where  X.  is  the  level  of  the  i-th  factor  in  the  u-th  observation.  He  defined 

1U 

the  standardized  variable  x,  as 

iu 


iu 


=  (X 


iu 


*i>/ 


He  then  took  the  design  problem  tc  be  as  follows: 


(a)  the  experimenter  is  to  specify  X.,  the  "center"  of  the  design  and 
scale  multiplier  for  each  variable, 


(b)  the  designer  of  the  experiment  is  to  choose  an  array  of  standardized 

levels,  x.  , 
iu 

levels  being 


levels,  x^,  at  which  the  observations  are  to  be  taken,  the  actual 


x.  «  X  +  x,  S. 
iu  i  iu  i 


In  other  words,  the  "center"  of  the  design  and  the  "spread"  are  specified  by 
the  experimenter  and  the  only  problem  of  the  designer  is  to  choose  the  x^ 
which,  of  course,  satisfy 


N 

£ 

u=l 


iu 


0, 


N 

£ 

u=l 


N 


I  shall  comment  on  this  basis  later,  but,  for  the  present,  will  indicate  the 
subsequent  developments.  In  the  case  of  the  first  order  designs,  the  criterion 
was  optimum  estimation  of  the  coefficients  in  the  equation 

y  s  +  P,x.  +  f5-,*,  +  .  .  .  +  (5,  x, 

'u  r0  rl  lu  2  2u  Kk  ku 
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axiu  the  optimum  design  is  one  in  which  the  x.  are  given  by  the  columns, 

l/2  1U 

after  the  first,  of  a  matrix  N  '  O,  where  O  is  an  orthogonal  matrix  whose 
first  column  consists  of  unit  elements.  Box  then  noted  that  if  the  number  of 
observations  is  k+1,  the  experimental  points  are  the  vertices  of  a  regular 
dimensional  simplex.  He  also  noted  that  any  rotation  of  this  regular  figure 
would  satisfy  the  conditions.  Box  and  Hunter  (1957)  developed  inconsiderable 
detail  the  concept  of  rotatability.  A  design  is  said  to  be  rotatable  if,  when 
the  levels  of  the  variables  are  standardized  as  stated  above  to  be 


(x, , 


‘2’ 


y  at  a  point  (Xj.x^,  .  .  .  ,  xfc) 
In  other  words  if  one  were 


,  x  ),  the  variance  of  the  predicted 

k  2 
is  a  function  of  these  x's  only  through  2 

to  construct  contours  of  variance  of  the  predicted  y  they  would  be  spherical 
with  center  at  the  'center'  of  the  design,  when  plotted  in  standardized  levele. 
They  stated  their  aim  to  be  "to  develop  arrangements  which  generate  infor¬ 
mation  (equal  to  the  reciprocal  of  the  variance  of  prediction  of  y)  symmet* 
rically  in  those  coordinates  regarded  as  most  relevant  to  the  experimenter.  " 
Box  and  Hunter  developed  second  order  designs  in  2  dimensions  by  taking 
two  or  more  concentric  rings  of  points,  with  each  ring  being  a  regular 
figure,  for  example  a  pentagonal  design  with  extra  center  points.  For  3 
dimensions,  they  took  points  equally  spaced  on  a  sphere,  for  instance,  by 
combining  a  regular  tetrahedron,  a  octahedron,  and  a  cube  with  additional 
center  points.  For  more  than  three  dimensions  they  suggested  the  combina¬ 
tion  of  the  points  of  a  2k  factorial,  and  2k  points  of  an  axial  set  and 
additional  center  points.  Throughout  attention  was  paid  to  the  problem  of 
blocking,  that  is,  of  arranging  the  totality  of  points  in  subsets  to  enable  the 
eliminacion  of  heterogeneity  between  the  units.  Box  and  Behnken  (1960a) 
developed  designs  by  operating  in  a  simple  way  on  first  order  simplex 
designs.  If  the  points  of  the  simplex  design  are  regarded  as  vectors,  one 
can  develop  additional  points  by  forming  sums  of  the  original  vectors  two 
at  a  time,  sums  of  the  original  vectors  three  at  a  time,  and  so  on.  The 
configurations  so  developed  are  then  scaled  to  satisfy  the  scaling  and 
rotatability  conditions.  In  this  way  they  obtained,  for  instance,  designs 
to  examine  4  variables  in  two  blocks  of  22  observations,  5  variables  in 
two  blocks  of  26  observations,  6  variables  in  two  blocks  of  34  observations, 

7  variables  in  two  blocks  of  33  observations.  The  last  one  in  this  list  is 
quite  impressive  in  that  it  usee  only  3  levels  of  each  factor  and  enables  all 
36  coefficients  of  a  second  degree  fitting  to  be  evaluated  reasonably.  It 
is  curious  that  all  the  points  except  the  center  points  be  on  a  hypersphere 
of  radius  ^3  (in  the  standardized  units).  Box  and  Behnken  (1960b)  developed 
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another  series  of  3-level  rotatable  designs  by  utilizing  incomplete  block 
configurations.  The  simnlest.  evamnle  w as  the  fnllnu/inor.  We  have  the 

—  A  *• 

balanced  incomplete  block  configuration 


'Block' 


If  a  "block"  contains  and  Xj,  it  is  replaced  by  the  4  treatment 

combinations  on  x^  and  x^,  (-1,-1),  (-1,1),  (1, -1)  and  (1,1),  the  other 

variables  being  taken  at  the  zero  level.  Bose  and  Draper  (1959),  Draper 
(1960a)  and  others  have  constructed  classes  of  second  order  rotatable 
designs.  Gardner,  Grandage  and  Hader  (1959)  and  Draper  (1960b,  1961, 

1962)  have  developed  third  order  rotatable  designs.  Throughout  it  appears 
that  the  designs  are  based  on  the  combination  of  symmetrically  placed  points 
on  spheres  in  the  standardized  factor  space.  The  ideas  of  Box  have  led  to 
the  development  of  a  considerable  array  of  designs,  all  based  on  the  concept 
of  rotatability.  Many  of  the  designs  are  remarkable  in  that  they  allow  the 
fitting  of  functions  of  the  second  or  third  degree  with  relatively  low  redun¬ 
dancy  of  experimental  points.  Also  by  choosing  odd  moments  up  to  partic¬ 
ular  order  equal  to  zero,  one  can  prevent  bias  in  the  regression  coefficients 
from  third  order  coefficients  in  the  polynomial  representation. 


The  motivation  for  the  development  of  the  array  of  rotatable  designs 
seems  to  be  summarized  by  Box  and  Behnken  (1960a,  page  840)  in  the 
following  quotation, 


"At  a  particular  stage  we  are  interested  in  the  behavior  of  the 
response  function  'in  the  neighborhood'  R  of  some  particular 
point  P.  We  have  in  mind  that  the  operability  region  O,  that 
is  the  region  in  the  space  of  the  yariables  in  which  experiments 
could  be  conducted,  is  fairly  extensive  and  that  P  is  not  close 
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to  the  boundary  of  O.  We  suppose  that  the  neighborhood  of 

•  •  1-1  i  T-»  j . .  r»  ...t:  -i.  — «  ...l  ...  <•  *u  ^ 
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boundary  of  O  and  that  scales,  metrics  and  transformations 
are  chosen  either  implicitly  or  explicitly  such  that  R  is  very 
approximately  spherical  and  is  centered  at  P.  " 

Essentially  all  the  designs  whose  development  I  have  mentioned  earlier 
were  aimed  at  controlling  the  variance  of  the  prediction  based  on  the  fitting 
of  a  polynomial  of  the  first  second  or  third  degree.  There  had  been  some 
attention  to  the  bias  in  estimated  polynomial  coefficients  from  higher 
polynomial  terms  that  were  ignored  in  the  fitting.  Box  and  Draper  (1959) 
made  a  direct  attack  on  the  problem  of  bias,  within  the  framework  of  previous 
developments.  The  situation  considered  was  that  a  function  f(x^,x2>  ■  •  •  , x^) 

is  fitted,  when  the  true  functional  dependency  is  . .  .  .x^).  The  mean 

square  error  of  a  prediction  consists  of  the  variance  plus  the  square  of  the 
bias.  Box  and  Draper  consider  the  average  over  a  region  of  interest  R  in 
the  (x^.x^, .  .  .  ,  x^)  space  of  these  two  components,  for  the  particular  case 

when  f(x^,x2 . x^)  is  linear  and  gjx^.x^,  .  .  .  ,  x^)  is  quadratic.  They 

conclude  that  the  optimal  design  is  very  nearly  that  which  would  be  obtained 
if  variance  is  ignored  and  only  bias  is  considered.  If  this  conclusion  is 
accepted,  it  would  appear  that  the  whole  class  of  rotatable  designs  based 
on  variance  considerations,  need  careful  re 'examination  from  the  viewpoint 
of  bias.  The  development  depends  strongly,  it  would  appear,  on  the  choice 
of  the  region  of  Interest  as  being  spherical  in  the  standardized  variables, 
and  on  equal  weighting  over  the  interior  of  the  "sphere"  of  interest.  The 
reasons  for  choosing  this  framework  appear  to  be  mathematical,  in  that  with 
this  framework,  integrals  can  be  evaluated.  Box  and  Draper  prove  a  theorem 
that  is  highly  indicative  of  the  nature  of  the  problem,  The  theorem  states 
that  if  a  polynomial  of  degree  d^  is  fitted  by  least  squares  over  any  region 
of  interest  R  in  the  k  variables,  when  the  true  function  is  of  degree  d^, 
greater  than  d^,  then  the  average  squared  bias  over  R  is  minimized  by 
making  the  moments  of  order  up  to  d^  +  d^  equal  to  the  corresponding 

moments  of  a  uniform  distribution  over  R.  So  if  one  knew  nothing  about  the 
true  function  except  that  it  can  be  represented  oy  a  polynomial  of  indefinitely 
large  depree  one  should  spread  the  observations  evenly  over  the  region  R. 
Clearly  the  definition  of  the  region  R  should  be  made  in  terms  of  variables 
for  which  one  could  hope  that  a  low  degree  polynomial  would  give  a  good  fit. 
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The  whole  line  of  development  appears,  however,  to  suffer  from  some 
defects  which  are  illustrated  by  the  simplest  designs  that  were  developed 
--  the  simplex  first  order  designs.  For  the  case  of  3  variables  with  4 
observations,  Box  exhibited  two  designs  which  he  claims  to  be  equally  good: 


X1 

X2 

X3 

X1 

X2 

X3 
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with 

D'D  =  D'D  =  41 
a  a  b  b 

where  I  is  the  3x3  identity  matrix.  These  two  designs  have  the  same 
center  and  have  equal  spread  with  the  definition  of  Box.  However,  if 
design  can  be  used,  x^  can  be  varied  between  -V" 2  and  7*2,  x^  can 

be  as  large  as  2—  ,  and  x^  can  be  as  large  as  ,  whereas  in  design 

Da  the  limits  for  each  x  are  from  -1  to  +1.  If  however,  the  situation  is 
such  that  one  can  vary  the  x's  over  the  ranges  specified  in  design  D  ,  one 
would  be  foolish  in  not  varying  them  over  the  same  range,  with  the  first 
order  design  D&,  and  if  one  does,  the  resultant  design  D£  ,  say,  is  clearly 

better  as  a  first  order  design  than  the  design  D^.  The  same  criticism  has 
been  made  by  Kiefer  {1961b). 

This  simple  example  brings  to  light  one  of  the  basic  problems  of 
exploration,  as  opposed  to  optimum  seeking,  namely,  that  the  region  of 
possible  experimentation  must  be  defined  if  one  is  to  attempt  to  develop 
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a  good  design.  The  simple  example  above  shows  that  the  standardisation  of 
variables  in  terms  of  root  mean  square  HevisHnn  r\<  i»y~l§  result!  in  peculiar 
restrictions.  It  would  seem  more  natural  and  appropriate  to  define  the  region 
of  possible  experimentation  in  terms  of  the  original  unstandardized  variables. 
It  one  is  exploring  the  relationship  of  a  yield  variable  y  to  a  single  control 
variable  X,  a  natural  restriction  would  be  that  one  can  experiment  at  X 
values  in  a  prechosen  interval  of  X,  say  from  X  =  a  to  X  =  b.  If  one  has 
two  control  variables  and  X^,  a  possible  specification  of  the  region  of 


permissible  experimentation  would  be  X  in  the  interval  (a  ,  b  ),  and  X 

1  \  1  c* 

in  the  interval  (a  ,b  ).  It  is,  of  course,  quite  likely  that  as  soon  as  one 

u  fa 


has  more  than  one  variable,  the  region  of  possible  experimentation  will  not 
be  rectangular  in  the  variables  originally  thought  of.  It  is  Inconceivable 
that  one  will  be  able  to  develop  a  useful  theory  of  experimentation  for  an 
arbitrary  region  of  possible  experimentation.  It  does,  however,  seem 
reasonable  that  one  can  choose  "new"  control  variables  that  are  functions 
of  the  originally  thought  of  variables  so  that  the  region  of  possible  experi¬ 
mentation  in  the  "new"  variables  is  approximately  either  a  hypercube  or  a 
hypersphere.  At  least  in  this  way  one  can  set  up  a  mathematically  defined 
problem  for  which  one  can  hope  to  get  an  answer.  One  might  hazard  the  guess 
with  the  emphasis  on  sphericity  that  results  from  considerations  of  rotatability , 
that  the  rotatable  designs  will  prove  to  be  good  designs  in  the  case  when  the 
region  of  possible  experimentation  can  be  defined  to  be  spherical.  Some 
problems  of  allocation  for  polynomial  regression  within  a  spherical  region 
have  been  considered  by  Kiefer  (1961b)  and  are  discussed  below.  It  appears 
that  a  few  of  the  Box-Hunter  rotatable  designs  of  very  specialized  nature 
are  optimal  with  respect  to  two  of  the  possible  criteria.  However  the 
implications  of  the  scaling  in  the  Box-Hunter  rotatable  designs  are  obscure. 


It  appears,  then,  that  a  more  fundamental  approach  to  the  problem  of 
design  would  take  as  its  base  a  definition  of  region  of  possible  experi¬ 
mentation,  provided  by  the  experimenter.  It  is  then  necessary  to  formulate 
the  aims  of  the  experiment,  and  it  is  at  this  point  that  one  opens  a  Pandora's 
box,  because  of  the  multiplicity  of  partially  conflicting  aims  that  always 
occurs. 
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Since  the  beginning  of  the  tormai  development  cf  design*  there  has  been 
some  attention  to  optimality  of  design.  In  the  simple  case  of  linear 
regression  on  an  interval  it  has  been  known  for  decades  that  the  best  disposi¬ 
tion  of  resources  for  estimation  of  the  slope  is  to  place  half  of  the  observa¬ 
tions  at  each  end  of  the  interval.  In  the  case  of  comparisons  of  two  groups 
it  is  obvious  that  for  maximum  precision  of  the  group  difference  one  should 
have  equal  numbers  of  observations  in  the  two  groups.  It  is  also  obvious 
that  if  one  has  several  groups,  and  one  has  the  same  interest  in  all  possible 
differences  of  pairs  of  groups,  one  should,  with  homoscedasticity ,  have 
each  group  equally  represented.  Indeed  the  requirement  of  equal  interest 
forces  equality  of  representation.  The  classical  symmetrical  designs  for 
error  control,  such  as  randomized  blocks,  Latin  squares,  balanced 
Incomplete  blocks,  were  considered  good,  because  the  prime  interest  of 
the  experimenter  was  considered  to  be  estimation,  with  equal  interest  in  all 
the  treatments,  which  were  taken  to  be  fixed.  They  were  also  based  on  the 
idea  that  the  main  difficulty  of  experimentation  was  to  control  variability 
between  experimental  units,  and  that  variability  within  a  group  of  experi¬ 
mental  unite  was  a  monotonic  function  of  group  size. 

Work  on  optimality  of  design  was  done  early  by  Plackett  and  Burmin 
who  showed  that  the  orthogonal  2n  plane  or  fraction*  of  these,  euch  as 
those  based  on  Hadamard  matrices  were  optimal  in  a  useful  sense  for 
qualitative  main  effects  of  two-level  factor*.  Indeed  they  resulted  in  as 
efficient  estimation  for  each  single  parameter,  as  one  could  obtain  if  on* 
used  the  whole  of  the  experimental  resources  just  to  estimate  that  single 
parameter,  and  this,  really,  is  much  more  than  one  was  ever  entitled  to 
hope  for.  A  few  years  later  optimality  of  design  was  attacked  frontally  by 
Elfving  (1952),  Chernoff  (1953)  and  Ehrenfeld  (1955).  The  topic  was  taken 
up  very  extensively  by  Kiefer  and  Wolfowitz  (1959)  and  Kiefer  (1958,  1959. 
1961a, b,  1962). 

The  whole  problem  of  optimal  design  is  of  course,  to  decide  what  to 
optimize  for.  Kiefer  (1959)  lists  several  possibilities: 

(a)  maximizing  the  infimum  of  power  of  test  of  &  null  hypothesis 
against  a  class  of  alternatives  (M-optimality), 

(b)  maximizing  the  limiting  power  of  test  in  the  neighborhood  of  the 
null  hypothesis  (L-optimality), 
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(c)  minimizing  generalized  variance  of  nf  p»r»rr’-itfT* 

(D -optimality) , 

(d)  minimizing  the  maximum  eigenvalue  of  the  variance-covariance 
matrix  of  estimates,  used  by  Wald  (1943)  and  Ehrenfeld  (1955) 

(E -optimality) , 

(e)  minimizing  the  trace  of  the  variance -covariance  matrix  of 
estimates  ( A-optimality), 

and 

({)  minimizing  the  maximum  variance  of  prediction  over  the 
experimental  region  (G -optimality). 

These  criteria  can  be  applied  to  the  totality  of  parameters  or  to  a  chosen 
subset  of  the  parameters  . 

It  needs  to  be  emphasized,  f  think,  that  all  these  criteria  are  related 
to  the  problem  of  control  of  error  with  a  model  which  is  assumed  to  be  true. 
It  is  not  clear  that  designs  which  are  good  for  error  control  are  also  good 
for  detection  of  bias  of  model,  as  Box  and  Draper  showed  in  work  that  I 
mentioned  earlier.  In  the  incomplete  block  problem,  for  instance,  I  am 
inclined  to  the  view  that  designs  which  have  some  repetition  of  treatments 
within  blocks  are  desirable.  Such  designs  will  be  inefficient  with  regard 
to  any  of  the  above  optimality  criteria,  if  balanced  incomplete  block  designs 
are  possible,  but  will  enable  better  examination  of  the  adequacy  of  the 
usual  additive  model. 

Kiefer  (1958,  1959)  has  proved  that  balanced  block  designs,  Latin 
squares,  Youden  squares,  orthogonal  arrays,  are  optimal  with  regard  to 
criteria  A.D.E  and  L.  These  results  are,  I  suppose,  of  some  mathematical 
interest,  and  suggest  that  if  one  has  a  balanced  array  of  experimental  units 
one  should  try  to  use  the  restrictions  of  the  array.  However  they  do  not 
answer  questions  like  whether  one  should  use  a  Latin  square  design  rather 
than  a  complete  block  design.  The  Latin  square  result  states  that  if  one  is 
going  to  use  the  Latin  square  model  for  analysis  one  should  use  the  Latin 
square  design,  and  as  such  is  not  at  all  surprising. 
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Kiefer  (1958,  p.  676)  characterises  M-optimality  as  "the  strongest  and 
least  artificial  of  the  four"  criteria,  D,E,  M  and  T.  »nd  it  *.vs.s  attoullon 
tv  tooting  of  nypotheses  that  led  Kiefer  to  give  the  examples  which  generated, 
apparently,  much  unnecessary  heat  at  the  Royal  Statistical  Society  meeting. 
Kiefer  pointed  out  that  if  one  had  6  observations  to  be  split  among  three 
populations  which  are  2),  i  *1,2,  3,  then  different  designs  were 

optimal  for  the  three  problems: 

(a)  point  estimation  of  0^,  6 2 ,  9, 

(b)  testing  the  hypothesis  =  = 

(c)  testing  the  hypothesis 

where  in  (b)  and  (c)  one  is  interested  in  alternatives  near  the  null  hypothesis. 
For  problem  (a)  one  should  take  2  observations  from  each  population,  for 
problem  (b)  one  should  take  one  of  the  populations  at  random  and  usa  all 
6  observations  on  it,  while  for  problem  (c),  one  should  take  two  of  the 
three  possible  populations  at  random  and  then  take  3  observations  from 
each.  This  example  shows  very  clearly  that  different  criteria  of  optimality 
can  give  radically  different  designs. 

The  work  of  Kiefer  and  Wolfowita  is  more  informative,  1  think,  in  the 
area  of  polynomial  regression  than  in  the  area  of  qualitative  experimenta- 
tion.  The  history  of  optimum  allocation  for  polynomial  regression  appears 
to  be  as  follows.  In  the  one -dimensional  case  for  which  the  units  can  be 
chosen  so  that  the  interval  of  experimentation  is  (-1,  1),  Guest  (1958) 
considered  the  G  criterion  above,  the  maximum  variance  of  a  prediction, 
and  showed  that  this  was  minimized  by  placing  of  the  points  at  each 

end  of  the  interval  and  at  the  zeros  of  the  derivative  of  the  k-th  degree 
Legendre  polynomial.  Hoel  (1958)  showed  that  if  one  wishes  to  minimi ae 
the  generalized  variance  of  the  coefficients  of  a  k-th  degres  polynomial 
the  optimum  allocation  was  the  same  as  that  obtained  by  Guest.  Kiefer  and 
Wolfowitz  (1959)  showed  that  the  beet  estimate  of  the  coefficient  of  xh, 
when  a  polynomial  of  degree  h  was  required  for  the  x-interval  (  *1  ,  1), 
was  to  place  l/2h  of  the  observations  at  each  end  of  the  interval  and  l/h  at 
the  points  cos(jn/h),  1  £  j  £  h  -  1,  which  may  be  termed  Chebychev 
spacing.  In  experimentation  on  the  square  -1^  x2^  *ke 
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best  test  of  interaction  term  x.x  is  obtained  by  placing  1/4  of  the 
observations  at  each  corner.  Of  course  all  the  above  solution*  depend  un 
the  total  number  of  observation*  being  appropriately  divisible.  Kiefer  (1959) 
gives  the  example  that  with  4  observations,  the  best  placement  for  cubic 
regression  on  the  interval  (-1,  1)  is  at  the  values  +1,  _+  1//5,  and  with  5 
observations  the  best  placement  is  at  the  values  0,  +  0,  5  11,  _+  1 .  The 
dependency  of  optimum  design  on  the  specific  value  of  N  is  avoided  by 
Kiefer  and  Wolfowitz  who  consider  how  best  one  would  place  an  infinite 
number  of  observations.  Such  placements  can  be  regarded  as  approximate 
designs,  and  they  proved  (I960)  a  rather  remarkable  theorem  that  the  design 
using  a  large  number  of  observations  which  minimizes  the  generalized 
variance  of  the  coefficients  of  a  polynomial  fitting  would  also  minimize  the 
maximum  variance  of  a  predicted  value  over  the  expei’imental  region.  It 
is  not  clear  just  how  useful  this  result  is  for  reasonable  numbers  of 
observations,  and  how  one  should  use  the  approximate  placing  given  by  the 
theorem  to  arrive  at  a  placement  for  a  reasonable  number  of  observations. 

With  this  proviso,  however,  this  later  work  of  Kiefer  and  Wolfowitz 
gives  an  indication  for  the  choice  of  design  in  "response  surface  exploration,  " 
at  least  if  one  views  the  matter  as  a  problem  of  polynomial  approximation. 

The  fact  that  the  generalized  variance  of  coefficients  is  minimized  would 
tend  to  indicate  (though  it  does  not  guarantee)  that  all  the  coefficients  of  a 
polynomial  are  being  estimated  with  reasonable  precision,  and  the  fact  that 
the  maximum  variance  of  a  prediction  is  minimized  should  to  a  moderate 
extent  permit  the  discovery  of  lacx  of  fit  by  the  polynomial. 

In  the  case  of  quadratic  regression  on  a  hypercube  bounded  by  >1  and 
1  in  each  direction,  in  q(«  2,3,4,  or  5)  dimensions,  Kiefer  (1961)  shows 
that  the  best  "infinite"  design  is  to  assign  a  proportion  a  of  the  expert* 
mental  points  to  each  of  the  23  corners,  a  proportion  (J  to  the  mid  point 

of  each  of  the  q2^  *  edges,  and  a  proportion  y  to  the  center  of  each  of 

the  q(q*l)2^~^  2-dimensional  faces  of  the  hypercube.  In  the  case  of  q 
equals  5,  the  values  of  a,  (3,  y  are 

a  =  .01928 

[S  =  .0003125 

V  «  .004475 
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However,  in  view  of  the  fact  that  the  a  set  contains  32  points,  and  the 
P  and  y  sets  contain  80  points  each,  this  "infinite  resources"  answer  is 
not  really  useful.  It  does  not  tell  us,  for  instance,  how  we  should  place 
say  50  or  60  observations.  It  does  appear  to  indicate,  however,  that  if 
the  G  criterion,  which  seems  a  somewhat  superior  one  for  exploration, 
is  adopted,  then  the  experimental  points  should  be  placed  near  the  corners 
and  edges  of  a  rectangular  experimental  region.  This  is  in  considerable 
contrast  to  the  rotatable  designs  discussed  earlier,  which  seem  to  devote 
much  attention  to  the  center  and  interior  of  the  region. 


Later  Kiefer  (1961b)  examined  polynomial  regression  when  the  region 
of  experimentation  and  interest  is  the  hypersphere  or  "ball,  11  S  1  . 

It  might  be  expected  that  the  designs  he  would  get  would  be  related  to  the 
rotatable  designs  in  that  the  latter  seem  to  be  aimed  at  a  epherical  region 
of  interest.  Kiefer  considers  the  approximate  case,  that  is,  the  "infinite 
resources"  case,  so  that  D-optimality  and  G-optim&llty  are  equivalent. 

He  was  able  to  characterize  partially  the  approximate  optimal  design,  and 
showed  that  it  iB  rotatable.  In  the  case  of  linear  polynomial  fitting,  the 
best  design  has  equal  weight  at  the  vertices  of  an  inscribed  regular  simplex 
or  the  vertices  of  any  other  inscribed  regular  polygon,  So  for  this  case  the 
maximally  spread  simplex  design  of  Box  (1952)  is  optimum  with  these 
criteria,  Also  in  two-dimensions  with  quadratic  .'regression,  the  design  with 
one  observation  at  the  center  and  one  at  each  vertex  of  an  inscribed  regular 
pentagon  is  D-optimal  and  hence  G-optimal,  However,  apparently  most  of 
the  rotatable  designs  do  not  have  these  optimality  properties ,  I  cannot 
regard  the  lack  of  optimality  properties  as  seriously  as  apparently  Kiefer 
does.  Kiefer  ( 1 9 6lb ,  p,  398),  feels  that  he  Justified  for  the  first  time  the 
use  of  rotatable  designs  but  I  regard  his  reeults  as  mathematically  rather 
elegant,  and  not  totally  relevant  to  the  problems  of  the  experimenter.  The 
repiesentation  of  yield  as  a  polynomial  in  the  control  variables  is  un&es- 
thetic  and  uneconomical  of  pa:ar.ic*c  r  a ,  ,  xcept  in  the  optim4  zation  problem , 
Even  In  the  optimization  problem  it  is  highly  questionable  whether  one  should 
do  local  experimentation  other  than  to  get  gradients.  I  agree  with  Kiefer 
that  the  framework  within  which  Box  and  his  associates  have  worked  has 
serious  logical  deficiencies,  but  also  have  the  view  that  they  developed  some 
very  useful  designs  and  design  ideas. 


CONCLUSIONS  ON  THE  EXPLORATION  PROBLEM.  The  problem  of 
studying  the  dependence  of  a  yield  variable  on  control  variables  is  not  well- 
defined.  Experimenters  with  this  problem  will  have  a  multiplicity  of  aims, 
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such  as  to  obtain  reasonably  precise  estimates,  reasonable  strength  of 
evidence  against  particular  null  hypotheses  of  interest,  ability  to  select 
a  functional  form  that  represents  the  data  well  and  is  economical  of 
parameters,  and  so  on, 

The  theoretical  statistician  can  obtain  optimal  designs  only  by  forcing 
the  problem  into  a  highly  idealized  simplified  form,  and  there  is  a 
tendency  to  regard  the  optimal  design  tor  idealized  simplified  form  as  the 
design  the  experimenter  should  use.  This  attitude  seems  to  be  exemplified 
by  Kiefer's  remark  (1959,  p.  316),  "Why  not  think  in  terms  of  the  right 
space  of  decisions  from  the  outset?"  I  have  yet  to  meet  an  experimenter 
whose  aims  can  be  represented  by  a  space  of  decisions,  which  is 
sufficiently  well-defined  to  be  susceptible  to  such  an  attack. 

The  work  of  the  optimizers  is,  however,  valuable,  because  it  gives  us 
suggestions  of  reBpects  in  which  a  plan  may  be  weak.  The  upshot  for  me 
of  the  work  1  have  reviewed  is  exemplified  by  the  following  cases;  In  the 
case  of  3  factors  in  a  cubic  region  (-1,  1),  I  would  do  the  following: 

3  -1 

(i)  with  4  observations  I  would  take  a  2  factorial  at  the  corners; 

(ii)  with  9  observations  1  would  use  a  1/3  replicate  of  the  3 
with  levels  -1,  0  and  1  for  each  factor; 

(iii)  with  15  points  I  would  use  the  corners  and  center  of  each  face 
and  the  center  which  is  essentially  a  central  composite  design 
but  not  rotatable; 

(iv)  with  27  points  I  would  use  the  full  3^  factorial  with  levels 
-1,  0,  and  1. 

If  the  problem  is  really  one  of  studying  the  dependence  I  would  try  to  per¬ 
suade  the  experimenter  to  do  the  full  factorial  (iv),  because  it  would 
enable  me  to  think,  to  some  advantage,  about  representations  other  than 
by  a  polynomial.  In  the  case  of  4  factors,  I  would  think  with  a  low 
number  of  possible  observations  in  termB  of  main  effect  plans  with 
observations  at  the  corners.  If  more  were  possible  I  would  consider 
the  sets  of  points: 
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S1 

:  (  +  1,  +,  1,  i  1,  +  1) 

S2 

:  (+1,  +1,  +1,  0)  with  permutations 

S3 

:  (+  1,  +  1,  0,  0)  with  permutations 

S4 

:  (+  1,  0,  0,  0)  with  permutations 

and  Sc 
b 

:  (0,  0,  0,  0)  . 

I  would  take  a  combination  of  these  sets  ,  For  instance ,  if  1  were  allowed 
24  points,  I  would  use  and  S4,  and  with  40  points  I  would  use 

and  and  so  on  [cf.  De  Baun,  1959]  .  Obviously  my  views  have  been 

influenced  by  both  Box's  work  and  by  Kiefer's  work, 

It  is,  however,  also  obvious  that  a  realistic  procedure  should  take 
account  of  sequential  plans,  Consider,  for  example,  the  investigation  of 
the  dependence  of  a  yield  variable  y  on  a  control  variable  x  in  (-1,  1). 
Suppose  that  the  information  on  y  for  each  chosen  x  is  available  as  soon 
as  the  experimental  run  has  been  made.  A  rational  procedure  is  not  to  use 
Chebychef  spacing  or  Legendre  spacing,  but  to  take  an  observation  at  xs-1 
and  at  xa+l.  One  would  then  take  one  at  x=0,  and  try  to  connect  three 
points  by  a  quadratic,  or  seek  a  reasonable  transformation  (non-linear)  of 
the  x  scale  so  that  the  3  observations  fell  on  a  line.  One  would  then 
probably  take  additional  observations  in  the  middle  of  the  gaps  of  the  best 
picture  one  has  obtained  up  to  the  time  of  planning  new  observations,  One 
would,  of  course,  have  prefaced  the  whole  matter  by  obtaining  a  rough 
idea  of  experimental  error.  It  is  very  difficult  to  see  how  the  concepts  of 
decision  theory  and  testing  of  hypotheses  can  be  brought  to  bear  on  such  a 
proce as , 

It  is  clear  that  practical  optimum  designing  depends  on  more  ingre¬ 
dients  than  have  so  far  been  incorporated  in  the  theory.  What  one  should 
do  depends  crucially  on; 

(a)  what  use  will  be  made  of  incomplete  information? 

(b)  what  is  the  rate  of  leed-back  of  experimental  information? 
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(c)  will  the  experimenter  be  able  to  do  additional  experiments  to  fill 
in  gap*  in  inthrmation? 

*.«4 

(d)  how  valuable  is  information  to  the  experimenter  in  relation  to  time? 
[What  is  the  present  value  of  future  information?  This  will  of 
course  depend  on  what  the  future  information  is.  ] 

(e)  what  is  the  cost  of  experimentation?  The  simple  idea  of  a  fixed 
cost  per  observation  appears  to  be  relevant  at  best  only  in  some 
technological  studies, 

The  difficulties  of  constructing  a  theory  which  incorporates  these  aspects 
appear  to  be  very  great,  but  should  not  dissuade  us. 

FINAL  NOTE.  It  is  unavoidable  that  I  cannot  describe  the  results  of 
every  paper  in  the  field.  The  reference  list  gives  only  papers  referred  to 
and  much  good  work  is  not  discussed.  A  notable  example  is  the  work  of 
Scheffe"  (1963)  on  experimentation  on  a  simplex. 
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INTRODUCTION.  The  theory  of  dimensions  which  I  will  discuss,  is  con¬ 
cerned  with  the  relations  that  may  be  found  between  quantities  occuring  in 
nature  as  a  result  of  the  operations  which  must  be  performed  in  order  to 
measure  them.  Dimensions  are  things  like  inches,  pounds,  minutes,  or 
volts,  or  rather,  the  characterie Stic s  which  standard  measurement  units 
such  as  inches,  pounds,  minutes,  or  volts  characterize;  namely,  length, 
mass,  time,  or  electrodynamic  potential,  Physicists  and  engineers  have 
been  making  an  analysis  of  these  dimensions,  as  a  phase  of  every  problem 
for  many  years,  The  point  I  want  to  make  today  is  that  a  dimensional  anal¬ 
ysis  of  a  problem  should  be  even  more  important  to  a  statistician,  since  such 
an  analysis  can  reduce  both  the  sice  of  an  experiment  and  the  work  required 
to  analyze  it.  As  it  is  not  hard  to  show,  a  dimensional  analysis  could,  in  a 
given  problem,  reduce  the  sample  size  by  more  than  half.  In  fact,  in  the 
present  stage  of  development  of  the  design  of  experiments,  dimensional 
analysis  offers  greater  hope  for  reducing  the  cost  of  experiments  than  any 
further  refinements  in  construction  of  blocks,  replicates,  and  so  forth,  in 
addition  to  its  promise  toward  reducing  the  cost  of  an  experiment,  dimensional 
analysis  has  another  virtue  almost  equally  important.  That  is,  a  dimensional 
analysis  carefully  conducted  can  yield  a  great  deal  of  information,  which  would 
otherwise  be  unobtainable,  about  the  type  of  model  which  should  be  adopted 
in  planning  and  analyzing  an  experiment, 

Although  the  basic  ideas  in  dimensional  analysis  have  been  in  use  among 
physicists  and  engineers  for  over  a  century,  they  are  apparently  almost 
unknown  among  statisticians;  at  least  there  is  no  reference  to  the  subject 
in  the  index  of  the  Journal  of  the  American  Statistical  Association  or  any 
other  statistical  publication  or  textbook  that  I  am  acquainted  with, 

However,  the  theory  of  dimensions  has  profound  implications  in  the 
study  of  statistical  problems.  The  theory,  originated  by  Joseph  Fourier  [l] 
is  based  upon  the  observation  that;  to  quote  Fourier; 

"Every  undetermined  magnitude  or  constant  has  one  dimension 
proper  to  itself,  and  the  terms  of  one  and  the  same  equation  could  not 
be  compared  if  they  had  not  the  same  exponent  of  dimension.  " 
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Thus,  if  a  group  ol  variables  are  connected  in  a  linear  equation  involving 
coefficients  to  be  determined  by  a  multiple  regression  those  coefficients 
must  represent  quantities  whose  dimensions  are  such  as  to  give  the  same 
overall  dimension  to  every  term  in  the  equation.  Similarly  also,  for  equa¬ 
tions  of  higher  degree. 

Therefore,  when  linear  or  polynomial  expressions  are  selected  as 
models  for  the  design  or  analysis  of  an  experiment,  it  should  be  required 
that  any  coefficients  postulated  in  these  expressions  have  a  dimensionality 
which  bears  a  reasonable  interpretation  in  context.  However,  one  might 
justly  criticize  a  model  in  which  one  of  the  coefficients  were  required  to 
assure  the  dimension  of  cubic  tons  per  square  degree  dollar  (and  I  have  seen 
such  an  example).  If  we  apply  the  theory  in  a  more  detailed  way  wo  can  arrive 
even  more  exactly  at  the  type  of  model  which  should  be  appropriate,  and 
obtain  information  concerning  those  interactions  which  are  to  be  expected 
and  which  can  be  ruled  out. 

An  example  will  serve  to  illustrate  what  dimensional  analysis  can 
provide  the  statistician.  In  Duncan,  2,  one  finds  an  experiment  in  which 
cotton  yarn  specimens  are  tested  for  yarn  strength,  yarn  length,  fiber 
tensile  strength,  and  fineness.  Slide  No.  1. 

X^i  Yarn  Strength,  Pounds 

X£:  Fiber  Length,  Inches 

X^:  Fiber  Tensile  Strength,  Pounds  per  square  inch 

X^:  Fiber  Fineness,  MicrogramB  per  inch. 

This  problem  is  discussed  and  analyzed  as  one  Involving  one  dependent,  anu 
three  independent  variables.  However,  as  a  result  of  dimensional  analysis, 
one  is  able  to  postulate: 

ffXlX3 

I  X2 
\  x4  i 

where  an  univariate  relationship  exists  between  the  quantities  on  the  right 
and  left,  An  analysis  of  the  data  is  shown  in  figure  1.  Using  the  method 
of  least  squares,  and  the  data  on  page  674,  one  obtains  the  regression  line: 


X2X3 
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X1X3/X4  =  •  05872  (X2  X3/X4)  ‘  3,90 

with  a  coefficient  of  correlation  of  r  =  .  955.  Applying  thia  formula  to 
another  set  of  data  from  the  same  source  given  on  page  699i  and  compar¬ 
ing  predicted  with  actual  values  of  X^,  one  has  a  sum  of  residuals  of  107, 
and  a  standard  error  of  9.  86.  Comparable  results  using  the  multiple 
regression  equation  given  on  page  693  are  114  for  the  sum  of  residuals  and 
8.  22  for  the  standard  error. 

The  value  of  the  dimensionless  equation  is  appreciated  by  considering 
that  it  contains  only  two  fitted  constants  as  against  four  for  the  multiple 
regression  equation  and  yet  predicts  approximately  as  well,  Moreover, 
the  calculations  were  vastly  simplified.  Finally,  by  keeping  the  numbeT 
of  fitted  constants  to  a  minimum,  one  avoids  the  danger  in  complex  predic¬ 
tive  hyper  surfaces  that  wild  contortions  may  occur  in  regions  which  do  not 
happen  to  be  represented  in  the  data,  yet  which  are  superficially  interpola- 
tive.  This  simplifies  and  improves  the  situation  from  every  point  of  view, 

In  general,  the  insights  provided  by  dimensional  analysis  are  valuable,  and 
the  method  is  easy. 

THEORY  OF  DIMENSIONS.  As  ia  shown  in  Murray  [3]  ,  any  primary 
dimension  which  is  effectively  present  in  an  experiment  or  process  can  be 
used  to  reduce  the  number  of  variables  by  one.  This  fact  is  explained  as 
follows;  External  standards  of  measurement,  such  as  an  international  metric 
unit  are  not  necessary  to  describe  a  process.  Any  quantity  within  the  proc¬ 
ess  itself  could  .  *■  ve  as  a  standard  of  measurement  for  other  variables 
measured  in  the  j~. ne  dimension.  In  any  formulae,  tables  or  charts  describ¬ 
ing  a  process  measured  in  this  way,  the  symbol  of  the  variable  taken  as  the 
mensurator  would  not  occur,  since,  being  the  standard,  its  value  would 
always  be  unity.  However,  am  outside  observer  could  convert  these  same 
formulae,  tables  or  charts  for  use  with  external  measurement  units,  by 
supplying  the  symbol  of  the  mensurator  as  a  denominator  under  the  symbol 
of  each  variable  to  measured. 

The  ratio  of  a  simple  or  compound  variable  to  its  mensurator  is  referred 
to  aB  a  dimensionless  term.  Since  we  reduce  the  number  of  variables  by  one 
for  each  primary  dimension,  m  variables  in  n  dimensions  can  be  represented 
in  the  form  of  m-n  dimensionless  terms  provided  an  adequate  system  of 
mensuration  can  be  found. 
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Each  variable  may  be  said  to  have  a  vector  of  dimension 


P 


i 

U 

m 


where  each  represents  the  exponent  taken  by  the  i  th  dimension  in 
the  dimensionality  of  the  variable  as  a  whole.  Thus,  if  i.  =  mass, 

*  l 

i_  =  length  and  i.  =  time, the  dimensional  vectors  of  speed  (meters 

^  -1  ^  1-2 
min.  )  and  pressure  (KG  meter  )  are; 


‘  0  * 

*r 

Speed  = 

1 

-1 

Pressure  » 

-2 

0 

The  dimensional  vectors  of  all  variables  that  can  be  relevant  to  a 
problem  forms  a  set  which  has  the  property  that  if  a  vector  P  belongs 
to  the  set  so  does  CP  where  C  is  selected  arbitrarily,  and  if  Pj  and  P^ 

belong  to  the  set,  so  does  P^  +  the  vector  of  the  product  of  the 

variables.  These  properties  define  a  linear  vector  space  which  is  a  closed 

set. 


If  we  can  find  n  variables  with  linearly  independent  vectors  in  this 
space,  these  variables  are  said  to  span  the  vector  space.  The  vector  of 
any  variable  can  be  duplicated  from  the  n  independent  vectors  by  scalar 
multiplications  and  vector  additions,  These  independent  vectors  are  a 
basis  for  the  vector  space  and  a  mensurator  for  any  variable  can  be  con¬ 
structed  by  combining  the  variables  having  these  vectors.  Any  n  vectors 
can  be  tested  for  independence  by  forming  the  determinant  which  has  these 
vectors  as  columns.  If  it  is  not  zero,  they  are  independent. 

Provided  then,  that  a  basis  of  n  independent  vectors  exists,  all  m-n 
variables  can  be  measured  by  mensurators  constructed  from  the  n  variables 
having  those  vectors,  Thus,  the  process  can  be  represented 
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where  each  term  is  composed  of  the  ratio  of  a  variable  to  its  mensurator. 

The  theorem  above  is  referred  to  as  the  Buckingham  Theorem  after 
Buckingham  i4|.  For  practical  methods  of  constructing  sets  of  terms  see 
Langhaar  [5],  or  Murphy[6j. 

The  completely  general  functional  expression  (1)  is  as  far  as  the  theory 
of  dimensionality  can  take  us,  The  explicit  function  must  be  determined 
by  experimentation  and  statistical  analysis,  or  from  subject  matter  theory, 
or  both.  When  m  -  n  =  1,  the  problem  is  solved  by  dimensional  analysis 
alone,  and  when  m  -  n  =  2,  simple  statistical  techniques  will  usually 
suffice . 

MIXED  DIMENSIONAL  AND  DIMENSIONLESS  EXPRESSIONS.  Previous 
texts  have  considered  only  cor  pietely  dimensionless  representations  and 
have  ignored  the  possibility  cf.  a  n-XItia?ly  dimension1 0 as  formulation.  Under 
these  circumstances  no  guide  e  '"as  provided  fev  t lie  analysis  of  problems 
in  which  the  vectors  of  the  variables  given  are  insufficient  to  span  the  vector 
space.  This  occurs  when  a  complete  specification  of  the  forces  acting  in  a 
process  cannot  be  made.  Such  incomplete  dimensional  specifications  do 
not  necessarily  negate  the  advantages  of  dimensional  analysis.  Some  of 
the  variables  may  still  be  reduced  to  a  common  mensurator,  thus,  permit¬ 
ting  some  reduction  in  the  number  of  variables.  For  example,  consider  a 
chemical  experiment  With  the  following  variables; 

X^  Amount  of  Yield,  Mols 

X ^  Amount  of  Reactant,  Mols 

X^  Amount  of  Acid,  Mols 

Temperature,  Degrees,  C 

Xj  Length  of  Reaction,  Minutes, 
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Obviously,  no  mensurator  can  be  found  for  X,  or  X..  Therefore,  a  com¬ 
pletely  dimensionless  expression  is  impossible  -  uxScnown  forces  have 
been  omitted  from  the  specification.  However,  X^  can  serve  as  a 

mensurator  for  X^  and  X^  permitting  the  formulation 


where  the  unit  of  length  is  the  length  of  X^,  or 


in  any  units, 

Therefore,  an  incomplete  dimensional  specification  reduces  our 
ability  to  condense  the  number  of  variables.  If  the  variables  are  all 
incomensurable  we  can  make  no  condensation.  If,  however,  some  of  the 
variables  are  commensurable,  we  can  reduce  their  number  to  the  extent 
that  commensurability  exists, 

A  CHEMICAL  WARFARE  EXAMPLE.  Thus  far,  we  have  described  a 
theory  which  offers  a  clear-cut  reduction  in  the  number  of  variables 
required  in  an  experiment.  Its  implications  are  so  plain  that  only  skepticism 
concerning  its  validity  would  be  grounds  for  ignoring  the  theory  and  benefits 
to  be  derived  from  Dimensional  Analysis. 

In  order  to  dispel  skepticism  concerning  the  theory,  I  have  applied 
Dimensional  Analysis  to  a  number  of  problems  in  various  fields  from 
which  data  was  available;  problems  in  Chemical  Engineering,  Agricultural 
Economics,  and  Quality  Control,  In  every  case,  the  Dimensional  Analysis 
accomplished  a  successful  reduction  in  the  number  of  variables  with  a  pre¬ 
dictive  value  equal  or  superior  to  any  previous  analysis  made  using  the  raw 
variables, 

One  application  was  in  the  field  of  assessment  of  the  coverage  capability 
of  toxic  chemical  ammunition  against  military  targets,  I  am  gratified  by 
the  results  obtained  so  far,  since  for  many  years  I  was  active  in  this  field 
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and  am  aware  of  the  high  potential  savings  that  would  result  from  any 
simplification  in  the  problem;  especially  any  model  which  would  eliminate 
or  reduce  requirements  for  testing  ammunition  over  wide  ranges  of 
meteorological  conditions. 

1  am  aware  that  much  theory  has  been  evolved  which  purports  to 
describe  behavior  of  toxic  clouds  in  the  atmosphere,  but  also  am  aware 
that  the  mathematical  complications  of  these  theories  are  such. that  actual 
models  for  purpose  of  prediction  rest  on  approximations  whose  accuracy 
is  uncertain,  and  which  do  not,  in  my  experience,  match  up  with  test  data 
obtained  in  the  field,  Dimensional  Analysis  cuts  across  this  theory  and 
leads  to  an  empirical  model  which  accounts  for  meteorological  factors 
more  satisfactorily  than  existing  models. 

To  illustrate  this  analysis,  Figure  No.  1  shows  the  variables  generally 
agreed  to  be  pertinent  to  the  problem  under  the  assumption  of  isotropic 
diffusion.  You  will  note  that  n,  the  Sutton  turbulence  parameter  enters  into 
the  problem  not  as  a  variable,  but  as  the  exponent  of  dimension  in  which 
the  diffusivity  is  expressed. 

The  temperature  is  omitted  from  this  list  since  there  is  no  completely 
agreed  manner  for  considering  it  and  it  does  not  fit  into  the  dimensional 
picture  here,  Sutton's  theory  ignores  it  and  it  is  customary  to  consider 
it  as  a  component  of  source  strength;  varying  the  effective  source  strength, 

Figure  1  shows  a  set  of  three  dimensionless  n  terms  which  according 
to  our  theory  should  be  able  to  replace  the  six  variables  shown  on  the  pre- 
vious  slide,  A  study  of  these  terms  shows  that  the  data  from  one  experiment 
in  the  field  could  be  used  to  predict  the  results  of  other  experiments  under 
different  meteorological  conditions.  Also,  it  implies  that  the  results  of  all 
conceivable  experiments  could  be  represented  by  a  single  surface  in  three 
dimensional  space,  or  as  a  family  of  curves  in  two  dimensional  space. 

Figure  2  shows  the  results  of  two  field  trials  plotted  in  the  space  of 
the  dimensionless  variables  shown  previously,  The  two  trials  were  con¬ 
ducted  with  the  same  type  of  shell,  and  at  approximately  the  same  tempera¬ 
ture.  However,  tho  wind  and  stability  conditions  were  considerably 
different,  and  therefore,  the  coverage  figures  obtained  were  also  consider¬ 
ably  different,  In  the  202  trial  on  the  left  the  wind  speed  was  1  meter/sec 
as  correspond  with  3.23  metera/sec  for  the  trial  203  the  right.  Stability 
was  moderate  inversion  for  the  trial  on  the  left  and  moderate  lapse  for  that 
on  the  right, 
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The  Sutton  parameters  n,  and  C  were  calculated  form  from  ths  wind- 
height  profiles  given  in  the  test  reports  using  the  Barad-Hilst  equations. 

As  the  chart  shows,  the  two  trials  were  sufficiently  different  to  pre¬ 
vent  any  overlap  between  the  two  families  of  curves.  However,  the 
critical  point  is  that  the  two  sets  of  observations  are  recogniaably  mem¬ 
bers  of  the  same  family  and  that  the  curve  -8,6,  which  occurs  in  both 
sets  of  data  matches  up  very  well:  in  fact,  a  line  projected  through  the 
two  points  obtained  in  trial  ^  202  passes  exactly  through  4  of  the  6  points 
shown  for  trial  / 203,  This  is  highly  encouraging  since  it  was  only  in 
meteorological  conditions  that  the  trials  were  different,  implying  that 
the  analysis  given  did,  in  fact,  satisfactorily  account  for  the  changes  in 
the  area  dosage  curve,  and  did  so  for  every  time  interval. 

We  infer  from  this  example  that  additional  tests  could  be  analysed 
to  fill  in  the  blank  spots  on  our  chart  and  an  empirical  equation  fitted  to 
this  data  with  ease,  since  only  three  variables  are  involved,  and  the  curves 
obtained  are  approximately  colinear, 

DIMENSIONAL  ANALYSIS  AND  MULTIPLE  REQRESION  ANALYSIS. 
Dimensional  Analysis  is  a  great  help  solving  the  difficulties  encountered 
in  multiple  regression  analysis.  It  has  several  advantages; 

a.  The  number  of  variables,  and  therefore  the  extent  of  the  calcula¬ 
tions  required,  is  reduced. 

b.  The  freedom  with  which  alternative  representationa  of  the  data 
can  be  formed  facilitate*  the  discovery  of  collinear  representations  which 
simplify  the  analysis. 

c.  The  predictive  equation  partakes  of  a  structural  validity  not 
entirely  dependent  on  statistical  estimation. 

The  value  of  the  dimensional  approach  may  be  appreciated  in  relation 
to  the  Bean  Ezekiel  graphical  method  of  multiple  curvilinear  regression 
analysis,  [?},  In  that  procedure,  no  explicit  mathematical  form  need  be 
ascribed  to  the  relationship  among  the  variables  but  by  an  iterative 
graphical  process  an  increasingly  accurate  approximation  to  the  curves 
involved  is  obtained,  and  the  result  is  a  set  of  charts  which  can  be  used 
directly  for  predictive  purposes,  or,  if  desired,  converted  to  tables, 
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nomographs  or  slide  rules.  A  scatter  plot  ot  residuals  is  «lI»u  obtained,  for 
an  estimate  of  error.  The  principal  drawback  of  the  method  was  the  frequent 
inability  of  the  analyst  to  isolate  recognizable  "draft  lines"  at  the  outset  due 
to  non-collinearity  of  response.  The  freedom  of  dimensional  representation 
should  largely  overcome  this  difficulty  and  increase  the  scope  of  the  method. 

CONCLUSION,  The  foregoing  exposition  has  shown  that  the  application 
of  dimension  theory  to  statistical  problems  can  result  in  valuable  Insight 
and  savings  in  experimental  design  and  analysis  and  should,  therefore, 
become  part  of  the  equipment  of  statisticians  generall.  Objections  to  the 
theory  have  at  times  been  advanced,  usually  on  the  basiB  of  special  examples 
wherein  functional  invariance  under  change  of  units  prevails  without  dimen¬ 
sional  homogeniety  (see  Bridgman,  [8).  However,  in  its  favor,  the  results 
obtained  by  Dimensional  Analysis  are  obtained  also  from  the  theory  of  partial 
differential  equations  as  applied  to  physical  problems  (see  Langhaar,  Chapter 
10);  the  theory  has  successfully  supported  the  researches  of  Maxwell, 
Rayleigh,  Helmholta,  and  others,  and  neither  the  literature  nor  the  experi¬ 
ence  of  the  present  writer  offers  an  instance  wherein  the  supposed  relation¬ 
ships  have  been  found  absent  in  fact. 

It  is  also  unclear  to  what  extent  the  standard  statisticsldesigns ,  tests, 
and  techniques  customarily  applied  to  dimensional  variables  can  be  applied 
to  dimensionless  variables.  Thus,  it  is  recognized  that  many  developments 
in  error  analysis  and  the  theory  of  sampling  will  be  required  to  exploit 
Dimensional  Analysis  to  its  fullest  (a  recent  paper  by  Halperin  and  Mantel, 

PI  would  appear  to  be  of  value  in  this  connection).  An  obvious  case  requiring 
attention  is  that  of  setting  confidence  limits  on  a  dependent  variable  which 
is  a  constituent  of  one  or  more  terms,  although  setting  limits  for  ths  tsrm 
themselves  would  be  straightforward 

It  is  hoped  that  being  made  aware  of  the  advantages  of  Dimensional 
Analysis,  statisticians  will  bend  it  to  their  needs  with  the  necessary  develop¬ 
ments. 
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THE  USE  OF  REGRESSION  ANALYSIS  FOR 

CORRECTING  FOR  MATRIX  EFFECTS  IN  THE  X-RAY  FLUORESCENCE 
ANALYSIS  OF  PYROTECHNIC  COMPOSITIONS 

Raymond  H.  Myers 

Virginia  Polytechnic  Institute:  Blacksburg,  Virginia 

and 

Bernard  J,  Alley 

U.  S,  Army  Missile  Command:  Redstone  Arsenal,  Alabama 

I.  INTRODUCTION.  X-Ray  fluorescence  methods  are  widely  used  in 
industry  for  the  analysis  of  a  variety  of  materials.  The  non-destructive  nature 
and  exceptional  speed  of  these  methods  are  largely  responsible  for  their 
widespread  use  and  increasing  acceptance,  The  direct  analyses  of  many 
materials,  for  example  can  be  accomplished  20  to  50  times  faster  than  by 
conventional  chemical  procedures.  This  allows  sufficient  time  after  an 
analysis  to  permit  the  correction  or  rejection  of  a  substandard  batch  of 
material  before  processing  is  completed. 

The  actual  X-Ray  fluorescence  method  of  analysis  may  be  briefly  described 
as  follows:  the  primary  beam  from  an  X-Ray  tube  impinges  on  the  surface  of 
a  specially  prepared  sample,  The  components  in  the  sample  surface  are 
immediately  excited  and  emit  their  characteristic  emission  lines  in  all  direc¬ 
tions.  Qualitative  analyses  are  made  by  determining  the  angles  at  which  the 
characteristic  emission  lines  from  the  sample  occur.  Quantitative  analyses 
can  in  general  be  performed  on  a  particular  component  of  a  mixture,  of  say 
K  components  by  positioning  the  radiation  detector  at  an  angle  which  corre¬ 
sponds  to  the  characteristic  emission  line  for  that  component  and  measuring 
the  emission  line  intensity.  The  Intensity  is  then  related  to  the  component 
percentage  by  a  suitable  calibration  procedure, 

The  intensity  ol  a  component's  characteristic  radiation  Is  not  a  simple 
function  of  the  concentration  of  that  component  alone  in  the  sample,  The 
intensity  depends  also  on  the  concentrations  of  the  other  components.  This 
is  caused  by  the  absorption  and  enhancement  among  the  components,  of  the 
primary  and  excited  radiation,  The  existence  of  these  interelement  or 
"matrix"  effects  is  one  of  the  more  serious  problems  encountered  in  X-Ray 
fluorescence  analysis  and  hence  inhibits,  to  a  great  extent,  the  use  of  this 
technique  as  a  quantitative  analytical  tool. 
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Many  non-mathematical  methods  have  been  devised  to  either  minimize 
or  correct  for  these  matrix  effects.  However,  they  have  been  found  to  be 
either  too  costly  or  too  time  consuming  on  samples  from  large  scale 
production  of  multicomponent  mixtures.  It  is  the  purpose  of  this  paper  to 
discuss  the  use  of  regression  analysis  in  the  correction  of  these  interelement 
effects  for  the  estimation  of  concentration  of  individual  components  in  a 
mixture  and  to  emphasize  the  application  to  a  particular  solid  rocket  propel¬ 
lant  mixture  in  current  use  by  the  U.  S.  Army  at  Redstone  Arsenal,  Huntsville  , 
Alabama. 

Effect  of  Solid  Particle  Size 


A  problem  which  may  be  encountered  when  one  is  analyzing  slurry  mix¬ 
tures  containing  solid  constituents  is  the  influence  of  solid  particle  sizes  on 
the  X-Ray  intensities.  It  might  be  necessary  that  any  analytical  procedure 
contain  some  type  of  correction  for  this  effect,  unless  of  course  the  individual 
particle  sizes  always  remain  constant  throughout  production.  Part  I  of  this 
paper  gives  the  analytical  technique  for  the  situation  in  which  the  particle 
sizes  were  experimentally  held  constant.  Part  II  extends  the  analytical  proce¬ 
dure  to  the  case  of  variable  particle  size. 

H.  ESTIMATION  OF  CONCENTRATION  (Particle  Size  Constant).  Samples 
of  a  five  component  solid  propellant  mixture  were  prepared  and  analyzed  for 
four  of  the  components.  (The  actual  ingredients  are  classified  and  hence  we 
shall  denote  them  in  the  text  as  components  1,  2,  3,  and  4  respectively). 

These  samples  were  taken  from  the  twelve  different  batches  in  a  narrow 
concentration  range  in  which  the  product  is  usually  manufactured.  The 
particle  sizes  of  the  solids  in  the  slurry  mixture  were  held  essentially  con¬ 
stant.  The  number  of  seconds  for  a  fixed  count  intensity  measurements 
were  recorded  in  rapid  succession  for  each  component.  The  same  was  done 

for  a.  synthetic  standard  sample.  The  response  variable  used  was  R=t  / 1  , 

s  c 

where  t  is  the  number  of  seconds  for  the  standard  and  t  the  number  of  seconds 
s  c 

for  the  component  in  question.  This  is  standard  procedure  used  in  this  type 
of  X-F.ay  work.  The  purpose  of  the  standard  and  the  subsequent  use  of  the 
ratio  of  the  standard  reading  to  the  unknown  reading  is  to  correct  for  elec¬ 
tronic  and  mechanical  changes  in  the  spectrograph.  The  data  is  found  in 
Table  I. 
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Consider  the  model; 
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where  is  the  intensity  ratio  for  component  i,  X^,  X^,  and  X^  are  the 

concentrations  of  the  individual  components,  the  B's  are  regression 

coefficients,  and  is  the  random  error  associated  with  R,,.  Note  that 
ij  !J 

the  concentrations  of  each  component  appear  in  the  model  despite  which  of 

the  four  ingredients  iB  being  detected.  Least  squares  estimates  of  the 

regression  coefficients  were  found  forthe  four  regression  lines,  These 

estimates  are  shown  in  Table  II  along  with  the  error  mean  squares  for  the 

regression  lines.  The  intensity  measurements  are  not  in  general  linearly 

related  to  concentration  but  in  the  reasonably  narrow  range  of  interest  shown 

in  Table  I,  a  linear  relationship  appears  to  hold  quite  well. 


TABLE  I.  Intensity  Ratio  Measurements  and  Composition  of 

Mixtures 


I 

R, 

(Compo 

sitions  in  weight  percent) 

R  X.  X. 

X, 

Batch 

_1 

2 

3 

4 

_1 

Z 

3 

4 

I 

t 

1 

1. 1240 

0.  8980 

0.  8219 

0. 9906 

0.  5514 

70,  18 

12.  53 

15.  04 

k 

k 

2 

0,  9285 

0. 8872 

0,  9308 

0. 9944 

0. 4426 

68.  84 

14.  26 

14. 75 

*- 

V: 

3 

1. 1214 

0.  8030 

0.  7668 

1. 1221 

0.  5631 

67.  51 

12.  79 

17.  39 

V' 

L. 

1 

4 

1. 1635 

0. 8706 

0. 9272 

0,  9832 

0.  5624 

67,  52 

14.  83 

15.  34 

1 

5 

0.  9415 

0.  8064 

0, 9026 

1,  1127 

0.  4505 

66. 10 

14.  52 

17. 0  3 

£ 

yr 

i: 

6 

0.  9039 

0.  8314 

0.  7596 

1.  0994 

0.  4425 

68.  86 

12.  30 

16.  72 

7 

1.  0712 

0. 8404 

0.  8662 

1.  0836 

0.  5290 

67.  34 

13.  95 

16.  35 

I 

8 

0.  9561 

0.  8731 

0.  8206 

1.  0290 

0.  4702 

69.  00 

13.  07 

15.  68 

9 

1.  0186 

0.  8431 

0.  8346 

1.  0591 

0.  5001 

68.  07 

13.  51 

16.  02 

h 

in 

10 

1.  0744 

0. 8124 

0. 7432 

1.  0967 

0.  5379 

68.  52 

12.  24 

16.  64 

11 

0.  9005 

0,  3320 

0. 8606 

1.  0798 

0.  4321 

67.  26 

13.  93 

16.  34 

i 

it 

»"■ 

12 

0.  9318 

0.  8913 

0.  8126 

0.  9880 

0. 4498 

69.  96 

12.  49 

14.99 

64 


Design  of  Experiments 


TABLE  II.  Regression  Coefficients  and  Error  Root  Mean  Squares 


Ingredient  1 

Ingredient  2 

s  =0. 00768 
e 

a  =0.  00776 

e 

b10  =  0. 15411 

b2,0=-L  437° 

b  *1,  8573 
*  1  * 

b  =0,01832 

*  i  * 

bj  2=-0. 00074 

b2  .-.0.03020 

b  -0. 00919 

A  >  * 

b  =0.02561 

c ,  5 

b,  =-0.00832 

1,4 

b,  =-0,  00790 
2, 4 

Ingredient  3 

Ingredient  4 

s  =0.  01130 
a 

s  =0,  01263 
e 

b3  Q  =  -l.  51670 

b.  n«0. 60788 
4,  0 

b  =-0. 07426 

5 ,  i 

b .  =-0.13257 

4, 1 

b3  2=0, 02008 

b .  ,=-0.00442 
4,  2 

b  3=0. 08024 

b .  ,o-0.  00641 
4,  3 

b  ’  =-0. 00328 

3,4 

b .  =0.05605 

4,  4 

We  can  ase  the  equations  in  (1)  to  develop  a  set  of  working  expressions  for 
estimating  the  concentrations,  i.  e.  , 

(2)  R_=b+BX_ 

where  R  represents  the  vector  of  intensity  ratios  and  b  the  vector  of  intercept 
terms.  The  bik  element  of  B  is  the  coefficient  of  X^  in  the  ith.  regression 

line.  X  is  the  vector  of  unknown  concentrations  that  one  seeks  to  estimate 
in  practice.  Inverting  (2),  we  have: 

(3)  X»B‘l(R  -  b)  . 

Here  we  have  a  case  of  the  use  of  a  set  of  simultaneous  multiple  linear 
regression  lines  in  reverse,  i.  e.  ,  inverting  the  regression  lines  to  estimate 
the  X's  i.  e.  ,  the  concentrations.  Williams  [3]  gives  a  discussion  of  the 
general  problem.  It  might  be  noted  that  the  concentrations  were  used  as  the 
independent  or  concomitant  variable  since  the  error  in  the  X's  is  very  small 
as  compared  to  that  for  the  X-Ray  intensity  ratios, 

Equation  (3)  represents  a  working  set  of  equations  for  estimating  the 
concentration  from  samples  from  running  production,  The  four  equations 
given  by  the  matrix  expression  in  (3)  are  as  follows; 
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=  -0. 14381+0,  54061  R,  +0.  07935  R?  -0,  08034  R^  +0.  08670  R4 

X  =38. 2619-0.  5767  R,  +42.5690  R,  -13.1116R,  +5.1478  R . 

2  12  3  4 

X  =8.  9016+0.  6984  R,  -10,  4829  R,  +15,  6926  R,  -0,  4547  R . 

3  12  3  4 

X  =-7. 1523+1.  3131  R, +2.  3448  R„  +0,  5705  R,  +18.  4010  R. 

4  12  3  4 

The  residual  errors  of  estimation,  calculated  from  the  original  data,  are 
shown  in  Table  III, 

TABLE  III,  Residual  Errors  of  Estimation  of  Concentration 


A 

a 

A 

A 

Batch 

xrxi 

VX2 

x3-x3 

VX4 

1 

-0.  0035 

0.  02 

-0. 19 

-0.  09 

2 

0.  0026 

0.  43 

-0. 14 

-0.  23 

3 

0,  0013 

-0.  01 

0 

0. 10 

4 

-0,  0026 

o 

o 

• 

0.  14 

0.  30 

5 

-0.  0026 

0,  16 

-0.  24 

0.  07 

6 

-0.  0026 

0,  03 

0.  06 

0,  07 

7 

0.  0027 

-0.  30 

0.  01 

-0.  31 

8 

0.  0046 

-0.  42 

0,  24 

0.  13 

9 

0.  0016 

0 

0.  12 

-0. 11 

10 

0.  0011 

0.  39 

-0,  06 

-0.  13 

11 

-0.  0014 

-0. 17 

0. 11 

0 

12 

-0.  0012 

-0. 14 

-0,  02 

0.  19 

CONFIDENCE  INTERVAL  ESTIMATES  ON  THE  CONCENTRATIONS.  Box 
and  Hunter  [l]  discuss  the  problem  of  joint  confidence  interval  estimates  on 
the  solution  of  a  set  of  simultaneous  equations  when  the  coefficients  are 
subject  to  error.  Their  work  was  actually  a  part  of  a  more  specific  problem 
of  finding  a  confidence  region  for  a  stationary  point  in  response  surface 
analysis.  However,  the  procedure  also  applies  to  our  problem  of  attaching 
confidence  limits  to  concentrations,  Suppose  that  in  general  we  have  K 
simultaneous  equations  of  the  type; 
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K 

iv  z  buxr0  iXBl>  ~c . «•) 

j*o  J 

where  the  b^  are  subject  to  error  (for  our  case  X^= 1).  Consider  the  quantities, 

K 

E  b  i  .=  6  (i=l,  2 . K), 

j  =  0  1J  J 

where  the  £  are  the  values  of  the  X'a  that  would  satisfy  (4)  if  the  actual 

regression  coefficients  were  used  in  place  of  the  b...  If  we  consider  a  vector 

xj 

of  the  6  ' s ,  say  6_  as  having  a  multivariate  normal  distribution  with  mean 

vector  £  and  variance -covariance  matrix  E(_6_6')aV,  then  the  expression 

-1  2  ~ 

£  'V  £  follows  a  X  distribution  [2]  with  K  degrees  of  freedom,  For  our 

case,  the  ith  element  of  £  can  be  written  as  where  Rj  is  the  estimate 

of  the  intensity  ratio  in  the  ith  regression  line.  For  estimates  of  the  elements 
of  V,  we  can  write 


h  1 


=vH- 


“d  c°v  (rA'  vv-hM 

n  l 


svH* 


where: 


s^=  sample  estimate  of  the  variance  of  R^  for  particular  values  of 


*r  ^  2 '  ^  3’  ^  4' 


s^=  sample  estimate  of  the  covariance  between  R^  and  R^. 

Ch^“(hl)  element  of  the  inverse  of  the  matrix  of  corrected  sums  of 
squares  and  products  of  the  X's  for  the  calibration  sample. 
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If  we  replace  the  elements  in  V  by  their  corresponding  estimates  and 
divide  by  the  appropriate  degrees  of  freedom  we  arrive  at  the  ratio 


n-8 

4 


s  z 

i  k 


ik 


lk 


which  is  distributed  as  F  with  4  and  n-8  degrees  of  freedom,  where  w  is 
the  (ik)  element  of  the  inverse  of  the  matrix  W,  the  matrix  of  residual  sums 
of  squares  and  products  of  the  R's.  We  can  write 


(5) 


■■VryVr 


where  the  X  are  the  estimates  of  the  concentration  obtained  from  equation 
(3),  If  we  replace  6^  by  the  expression  in  (5),  we  have 

}b,  bk,wik 

r4,„.r(^)  JJLii — !  1  1  1  u  “ 


H 


(6) 


ES(X.{)(M  )  q 

=,Sl8)  J  J  »  1  J* 


H 


where  q^  is  the  (jl)  element  of  the  matrix; 


QsB'W^B, 


Here  is  the  (ij)  element  of  the  matrix  B, 

Equation  (6)  represents  simultaneous  joint  confidence  Interval  estimates 
of  the  actual  concentrations  £.,  £  ,  £  ,  and  £  Thus  if  we  are  given  values 
A  A  A  1  X  i  4 

of  th'j  estimates  X^>  X,.  and  X^,  we  can  substitute  particular  values  of 
the  concentrations  £^,  £  ^ ,  £  j,  and  £^  into  equation  (6)  and  if  the  resulting 
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expression  is  less  than  F  (upper  tail) ,  then  those  values  of  the  £  's 

ci ,  4 ,  n  •  8 

fall  inside  the  1  rtflfl -n W*  confidence  bend. 

The  elements  of  the  W"*  and  Q  matrices  are; 

W_1  =  7214.8162 


-15.343  -274.4115  219.582 

2.4899  2,8058  0.77389 

10.8662  -3.6781 

5.  3264 

III.  VARIABLE  PARTICLE  SIZE,  An  experiment  was  conducted  in 
a  manner  similar  to  that  described  in  II  except  that  the  particle  size  was 
allowed  to  vary.  Components  2  and  4  are  the  only  ones  for  which  the 
particle  size  is  an  important  factor  in  its  effect  on  the  intensity  ratio 
measurement.  The  point  should  be  made  here  that  it  is  assumed  that  the 
particle  sizes  are  known  in  a  practical  situation,  i.  e.  ,  for  a  sample  of 
the  propellant  from  running  production  one  can  determine ,  from  the 
physical  source  of  components  2  and  4,  at  least  the  mean  particle  size. 

The  degree  of  difficulty  here  would  depend  upon  the  precision  with  which 
these  two  components  are  manufactured.  No  attempt  was  made  here  to 
consider  such  problems  as  particle  size  distribution,  Likewise  no  attempt 
was  made  to  consider  the  degree  to  which  the  particle  sizes  of  components 
2  and  4  are  altered  by  the  mixing  process  itself. 

A  l/8  fraction  of  a  2^  factorial  design  was  used  with  four  replications 
at  each  point  and  in  the  center  of  the  design,  The  factors  are  the  concen¬ 
trations  X^,  X^,  X^i  X^,  and  particle  sizes  and  W^,  Table  IV  gives 

the  design  matrix  and  the  defining  contrasts. 


2554.8459  -3201.5790  3439.8046 

4014.0983  -1714.2325  2122.4663 

2679.7650  -1867.4436 

2825. 4942 
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TABLE  IV. 

Design  Data  and 

Defining  Contrasts 

Batch 

Treatment 

X 

X, 

x. 

X. 

w. 

Combination 

_L 

_ 2 

_3 

_4 

2 

4 

1 

abef 

i 

1 

-1 

-1 

1 

1 

2 

cdef 

-l 

-1 

1 

1 

1 

1 

3 

(1) 

-l 

-1 

-1 

-1 

-1 

■1 

4 

ace 

l 

-1 

1 

"1 

1 

-1 

5 

bde 

-i 

1 

-1 

1 

1 

-1 

6 

abed 

l 

1 

1 

1 

-1 

-1 

7 

adf 

l 

-1 

-1 

1 

-1 

1 

8 

bef 

-i 

1 

1 

-1 

-1 

1 

9 

midpoint 

0 

0 

0 

0 

0 

0. 

Defining  contrasts:  I,  ADE,  BCE,  ACF,  BDF,  ABCD,  ABEF, 
CDEFr  (Particle  Size  Units  are  per  cent  fine  fraction  on  total  In¬ 
gredient  baeie) 


A  eet  of  multiple  regression  equations  of  the  type 


(8) 


R  a  £ 

«  k=0 


Bikxkj 


+Bi5W2J+Bi6W4j+'ij  <l"l'2'3'4> 


were  fit  to  the  design  data,  where  se  before  Xq»1,  Table  V  shows  the 

estimates  of  the  coefficients  of  the  regression  line  in  (8).  (8)  can  be 

written  as 


A 

R"IljX  +  B2W  . 


We  can  then  "correct"  the  intensity  ratio  vector  for  particle  siee  and 
solve  for  the  vector  a; 


(9) 


x  =  b1'1(r-b2w) 


. 
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T'nia  re  suits  in  the  following  “St  of  *mt«Hnn« 

A 

X^U,  998725Rj+598.  526R2+82.  076R3+395.  848R4-988.  676- 

13.  5897W.-2,  2816W .  , 

2  4 

A 

X_=8, 84359P-.  +  3207. 192R-+439.  287R,+2109.  9897R,, -5226.  75124- 

2  12  3  4 

74.  7551W  -11  4927W  , 

2  4 

X  =1,  653744R.+867,  2777R,+137,  368R.+576.  007R.-1437.  2059- 

3  1  2  3  4 

19.  37799W, -3,  02207W  . 

2  4 

A 

X  ■  3.0437R,+1258.126R_+17S.  7089R-+847 . 6307R  -207 3,  6127- 

4  12  3  4 

28.  46016W.-5.  32504W 
2  4 

The  equation  in  (8)  could  also  be  used  to  estimate  particle  siee  when  either 
the  particle  sise  cannot  be  determined  or  one  feels  that  the  mixing  process 
has  caused  sufficient  "grinding"  that  there  has  been  a  change  from  the 
particle  siees  of  the  pure  components.  Of  course  this  would  require  a 
chemical  analysis  of  two  of  the  components  of  the  mixture , which  of  course, 
it  time  consuming. 


TABLE  V.  Estimates  of  Regression  Coefficients  and  Error 
Root  Means  Squares  for  Equation  (8) 


ngredient  1 

Ingredient  2 

Ingredient  3 

Ingredient  4 

■0. 02005 

s  “0.  01199 

s  >0.  00830 

s  *0.02298 

e 

e 

e 

e 

10* ”4, 84!3 

b20»2. 82710 

b30«-8,4503 

b40«-8, 19590 

u»1.9320 

b21*-0.  03948 

b31»0, 11398 

b.,«0,  08438 

12«0.  05104 

b22«-0. 01436 

b32*0, 09337 

b42»0. 08200 

13»0. 06237 

b23»-0. 02355 

b^-0. 15847 

b.,»0, 08462 

4  3 
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I 


rj; 

$ 


< 


TABLE  V 
(cont'd.  ) 


Ingredient  1 
b,  =0.  05010 
b  *-0. 00582 
b16»0.  00024 


Ingredient  2 
b24=-0. 05424 
b25«0, 01072 
b26=-0.  00218 


Ingredient  3 

b.  =0. 07888 
34 

b,  =-0, 00682 
j  b 

b36=-0. 00245 


Ingredient  4 

b. .=0.14812 

44 

b  _B-0, 00815 

45 

b46=0. 00417 


Table  VI  shows  the  residual  errors  in  estimation  of  the  concentration  using 
equation  (9). 


TABLE  VI.  Residual  Errors  in  Estimation  of  the  Concentration 
“Using  Equation  (9^  (Units  in  wt. 


Batch 

Ingredient  1 

*rxi 

Ingredient  2 

VX2 

Ingredient  3 
CK3-X3) 

Ingredient  4 

(x4-x4) 

1 

-.006332 

.2216 

vO 

O 

O 

w — 4 

8 

0196 

2 

-. 008813 

.  3308 

-.1491 

-.0373 

3 

-. 013621 

.  5426 

-.2567 

-. 0684 

4 

.  003560 

-.0692 

,  0417 

0485 

5 

,  008362 

-.2675 

,1365 

-.0293 

6 

-.001603 

.  0227 

-.0061 

.  0097 

7 

. 004032 

0966 

.  0539 

-.0513 

8 

. 007943 

-.  258  3 

.1234 

-.0280 

9 

. 007812 

-.8720 

,  3427 

,  3712 

SUMMARY ,  A  set  of  equations  is  given  for  estimating  the  component 
concentration  in  a  certain  solid  propellant  mixture  in’ terms  of  the  X-Ray 
intensity  readings  of  each  component.  The  method  used  involves  inverting 
a  set  of  simultaneous  multiple  linear  regression  equations.  The  concentration 
of  each  ingredient  appears  in  each  equation  in  order  to  correct  for  "matrix" 
conditions  which  do  effect  the  X-Ray  intensities.  The  significance  tests  on 
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individual  components  indicate  that  these  interelement  conditions  do,  in 
fact,  exist  for  the  mixture  in  question,  Joint  confidence  regions  were 
developed  for  the  concentrations. 

Since  it  was  suspected  that  the  particle  size  of  pure  components  2  and 
4  also  effect  the  X-ray  Intensity,  a  linear  model  involving  particle  size 
was  fit  to  the  data  from  a  1/8  fraction  of  a  Z°  factorial  design.  This  did 
indicate  that  particle  size  was  in  fact  a  necessary  consideration  and 
resulted  in  a  set  of  equations  for  estimating  the  concentration  of  each 
component  In  terms  of  an  intensity  reading  which  is  adjusted  for  particle 
size . 
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SAMPLING  FOR  DESTRUCTIVE  OR  EXPENSIVE  TESTING 


Joseph  Mandelson 

Quality  Evaluation  Division,  Quality  Ansurance  Directorate 
Edgewood  Arsenal,  Maryland 


INTRODUCTION,  In  recent  years  the  engineer  has  been  impressed 
with  the  fact  that  the  principles  of  sampling  are  essentially  statistical  in 
character  because  the  effect  of  sampling  can  only  be  appraised  in  terms 
of  operation  of  the  laws  of  chance.  Consistent  with  this  revelation,  the 
engineer  by  and  large  has  been  content  to  retire  from  the  field  of  sampling 
and  abdicate  his  responsibilities  in  this  area  to  the  statisticians,  A  few 
hardy  souls,  confirmed  do-it-yourselfers,  took  it  upon  themselves  to  in¬ 
vade  the  statistical  field  and  learned  to  acquit  themselves  creditably  in 
the  area  of  sampling,  They  even  branched  out  into  other  aspects  of  statis¬ 
tics  germane  to  engineering.  However,  the  influx  of  engineers  into  the 
statistical  preserve  was  not  sufficiently  large  to  be  able  to  handle  the 
relatively  heavy  volume  of  activity  required,  Then,  too,  a  number  of  work¬ 
ing  tools  were  prepared  by  statisticians,  presumably  for  use  by  quality 
engineers  and  inspection  personnel,  to  cover  a  multitude  of  sampling 
problems  as  these  occur  in  quality  assurance,  Some  of  these  tools  are 
quite  complicated;  for  their  complete  comprehension  they  demand  more  in 
the  way  of  statistical  knowledge  on  the  part  of  the  would-be  user  than  the 
authors  are  prepared  to  admit,  As  a  consequence  there  is  a  degree  of 
obscurity  in  the  field.  The  engineer  is  urged  to  consult  the  statistician 
whenever  his  state  of  confusion  or  the  importance  of  the  matter  in  hand 
appears  to  warrant.  However,  the  engineer  should  long  algo  have  risen  in 
wrathful  protest  against  statistical  tools  supposedly  prepared  for  his  use 
but  which  he  finds  slippery  and  elusive  to  the  point  of  unintellgibility. 

Actually,  is  it  so  important  that  comprehension  of  the  mathematical 
derivation  of  statistical  methods  be  made  an  essential  prerequisite  to 
their  efficient  use?  Would  not  an  explanation  of  the  basic  factors,  in  non- 
mathematical  terms  followed  by  a  detailed  by-the-numbers  procedure  to 
use  in  the  given  context  suffice  ?  At  any  rate,  I  propose  to  try  this  approach. 

.SAMPLING  RISKS,  The  layman  has  long  regarded  the  field  of  sampling 
with  healthy  suspicion;  he  has  felt  in  his  bones  that  sampling  is  a  risky 
business  at  best.  The  well- public itetf  failures  of  public  opinion  polls  in 
predicting  the  results  of  crucial  election  weaken  his  confidence  in  statis¬ 
tical  methods,  His  instinct  in  regard  to  risks  is,  of  course,  entirely 


74 


Design  of  Experiments 


correct  as  his  everyday  experience  with  matters  governed  by  the  law  of 
chance  illustrates,  Allusion  may  be  made  to  games  of  change,  insurance 
anrl  the  like.  Much  can  be  Isamed  from  pertinent  analogies,  .Let  us  con¬ 
sider  examples  from  games  of  chance  such  as  bridge,  etc, 

A  well  shuffled  card  deck  is  analogous  to  a  lot  of  material  from  which 
a  sample  is  taken,  with  one  important  difference;  the  exact  composition 
of  the  deck  is  known,  that  of  the  lot  is  not,  Each  hand  in  bridge  is  a  sample 
of  13  from  a  lot  of  52.  A  hand  of  exactly  average  etrength  would  contain  one 
card  of  each  of  the  13  values  and  a  4 -  3 - 3 -  3  distribution  in. suits.  Our  experi¬ 
ence  tells  us  that  such  a  hand  is  almost  never  observed.  Instead  We  find 
that  some  hands  are  stronger  and,  by  the  same  token,  others  are  weaker  than 
the  average,  This  should  teach  us  that  a  sample  is  very  rarely  truly  indica¬ 
tive  of  the  composition  of  the  lot.  Instead,  we  find  that  the  sample  sometimes 
appears  to  be  better,  sometimes  worse  than  the  average,  if  by  average  we 
mean  a  sample  whose  composition  is  exactly  proportionate  to  that  of  the  lot, 
Further,  we  find  that  small  variations  from  the  average-strength  hand  are 
quite  frequently  encountered,  large  variations  are  relatively  rare, 

In  real  life,  the  composition  of  the  lot  is  almost  never  known,  the 
purpose  of  the  sample  is  to  permit  us  to  make  inferences  and  decisions  re¬ 
garding  the  acceptability  of  the  lot  sampled.  Since  we  recognize  that  the 
sample  rarely  reveals  precisely  what  the  true  quality  is,  it  must  be  accepted 
that  some  of  the  decisions  at  which  we  arrive,  based  on  results  obtained  in 
testing  the  sample,  may  be  in  error,  There  are  two  types  of  such  error. 

PRODUCER'S  RISK,  The  Type  I  error,  so  called,  is  the  decision  to 
reject  a  lot  which  is  really  acceptable,  This  occurs  when  the  sample, 
through  chance  variation,  indicates  a  larger  proportion  of  defectives  than 
that  which  is  really  present  in  the  lot.  It  is  the  equivalent  of  the  bridge  hand 
which  contains  almost  no  strength.  These  hands  occur  occasionally,  with 
predictable  frequency.  In  the  same  way  lots  of  acceptable  quality  will  produce 
a  sample  of  given  size  which,  with  predictable  frequency,  will  indicate  the  lot 
to  be  unacceptable,  It  should  be  noted  that,  while  the  frequency  of  such 
occurrences  can  be  predicted  (say  once  in  twenty  samples)  the  actual  event 
(which  one,  if  any,  of  the  twenty)  cannot  be  foreseen;  it  occurs  at  random 
intervals.  In  any  case,  the  rejection  of  an  acceptable  lot  occurs  with  a  cer¬ 
tain  probability  equivalent  numerically  to  this  predicted  frequency  of  the 
Type  I  error.  Since  a  rejected  lot  will  require  100%  inspection  of  the  lot, 
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rework  or  scrapping  cf  the  material,  it  is  plain  to  see  that  Ihe  risk  of  this 
unfortunate  occurrence  is  one  which  will  cause  the  producer  some  economic 
loss.  For  this  reason  this  is  called  the  Producer's  Risk, 

CONSUMER'S  RISK.  On  the  other  side  of  the  coin  we  have  the  Type  II 
or  beta  error  which  occurs  when  we  decide  to  accept  a  lot  which  is  really 
unacceptable.  This  occurs  when  the  sample,  by  chance,  yields  results  which 
happen  to  conform  to  the  requirements  which  decide  the  acceptability  of 
material  offered  him.  This  situation  is  analagous  to  the  bridge  hand  which 
is  abnormally  strong.  The  comments  already  made  with  respect  to  the 
Type  I  error  are  also  applicable  to  the  Type  II  error,  viz.  ,  the  frequency 
of  such  occurence  can,  within  reason,  be  predicted  if  certain  information, 
normally  not  available,  is  at  hand  or  can  he  assumed.  The  effects  of  the 
Type  11  error  are  quite  different,  of  course,  since  the  material  now  becomes 
the  property  of  the  user  and  the  excessively  high  proportion  of  defectives  it 
possesses  will  undoubtedly  cause  him  to  sustain  certain  kinds  of  loss.  The 
Type  11  error  gives  rise  to  the  Consumer's  Risk. 

EFFECT  OF  SAMPLE  SIZE  ON  RISK.  Both  types  of  error  and  the 
associated  risks  may  be  reduced  by  using  larger  samples.  It  can  be  shown 
that  the  amount  of  information  concerning  the  quality  of  the  lot,  available 
from  the  sample,  varies  as  the  square  root  of  the  numerical  size  of  the 
sample.  Consequently,  if  one  wishes  to  double  the  information  in  the  sample 
he  must  multiply  his  sample  size  by  four.  Clearly,  this  ca*i  soon  become 
an  expensive  business  and  leads  to  diminishing  returns. 

It  must  ever  be  kept  in  mind  that  the  risks  we  have  considered  have 
substantial  significance,  economic  and  otherwise.  Both  risks  lead  to 
various  types  of  loss,  many  (but  not  all)  of  which  can  be  measured  in  mone¬ 
tary  terms  and  all  of  which  must  be  assumed  either  by  the  supplier  or  the 
consumer.  Whether  these  costs  will  weigh  more  heavily  on  the  former  or 
the  latter  is  determined  by  the  quality  of  the  lot,  the  sampling  plan  and  the 
level  of  quality  specified.  The  risks  and,  therefore,  rheir  cost  can  be 
reduced  by  increasing  the  sample  size  but  this,  in  turn,  raises  the  cost  of 
sampling  and  test  which  is  customarily  borne  by  the  consumer.  We  are 
reminded  that  raising  the  sample  size  to  effect  an  arithmetic  increase  in 
information  will  necessitate  a  geometric  increase  in  the  costs  associated 
with  the  simple  size, 
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TOTAL  COST  OF  SAMPLING,  If  one  is  realistic  he  will  recognise  that 
the  total  cost  of  sampling  includes  not  only  the  cost  of  taking  and  testing  the 
sample  but  also  the  losses  occasioned  by  the  operation  of  the  risks  already 
discussed.  It  may  appear  strange,  perhaps  unbelievable,  that  there  should 
be  any  who  will  not  accept  the  fact  that  there  are  risk  losses  to  evaluate  and 
will  not  agree  to  include  these  in  the  reckoning.  But  these  doubting  Thomases 
are  like  their  predecessor  -  unless  they  see  little  green  bills  passing  over 
a  counter  from  one  hand  into  another  they  cannot  agree  that  a  cost  or  loss 
has  been  sustained.  It  is  particularly  unfortunate  when  such  short-sighted 
persons  get  into  a  position  where  they  are  able  to  influence  the  sampling 
plan  to  be  used.  When,  in  consequence,  losses  are  sustained  from  defec¬ 
tives  regarding  which  complaints  are  received  from  users,  and  from  lots 
unnecessarily  screened  or  reworked,  such  people  eloquently  display  newly 
washed  hands  as  tokens  of  their  freedom  from  sin  and  learnedly  discuss  the 
poor  inspection  job  turned  out  by  that  overly-large  and  over -paid  staff  of 
inspectors.  Now,  say  these  management  experts,  if  we  really  want  to  save 
money,  here  is  some  fat  which  can  be  advantageously  trimmed.  It  will 
never  occur  to  them  that  insistance  on  minimum  sample  sizes  reduces 
a  relatively  small  cost  but  incurs  much. larger  risks  which  require  the 
piper  to  be  paid  in  large  and  repeated  installments. 

The  true  total  cost  of  sampling  is  determined  by  several  parameters, 
chief  among  which  are  the  sample  size,  the  specified  quality  level,  and 
the  consumer's  and  producer's  risks.  There  are  other  parameters  involved 
in  the  final  result  such  as  the  cost  of  making  a  test,  the  true  quality  of  the 
lot,  the  cost  of  reworking  an  item  declared  defective,  etc.  For  our  pur¬ 
pose,  it  is  desirable  to  search  out  the  interrelationships  among  the  four 
parameters  first  mentioned, 

Clearly,  the  larger  the  sample,  the  more  costly  the  test,  At  the  same 
time,  the  risks  and  their  attendant  costs  are  reduced  by  large  samples  , 

This  situation  leads  naturally  to  the  supposition  that  there  may  be  some 
point  at  which  the  size  and  coat  of  the  sample  are  so  happily  related  to 
the  costs  of  the  corresponding  risks  that  the  over-all  cost  Is  a  minimum, 

The  size  of  sample  which,  within  the  stated  conditions,  brings  about  such 
a  desirable  result  may,  with  propriety,  be  designated  the  optimum  sample 
size.  The  existence  of  such  an  optimal  solution  can  easily  be  demon¬ 
strated  arithmetically  (2),  However,  there  are  some  matters  which  we 
should  clarify  before  venturing  further,  Theee  include  the  meaning  of  and 
ways  to  handle  the  cost  of  the  risks, 
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^woiLNG  THr.  r-ROD  UC  ER '  5  RISK.  The  producer  '  b  or  alpha,  risk  h  a  a 
already  been  described  as  the  risk  that  the  sample  may  indicate  the  lot  to 
be  unacceptable  when  it  is,  in  fact,  quite  acceptable.  If  the  test  is  non¬ 
destructive  or  the  cost  of  making  the  test  is  not  prohibitively  high,  it  is 
economically  possible  to  test  or  examine  each  item  in  all  rejected  lots. 

In  this  way  the  original  erroneous  decision  will  be  corrected  at  a  price  - 
the  cost  of  such  test  or  inspection  is  the  cost  of  rejecting  the  lot  and,  uniier 
these  circumstances,  the  price  paid  for  the  Type  I  error  is  relatively  low. 
But  if  the  test  is  quite  expensive,  particularly  if  it  damages  or  destroys 
the  item  tested,  it  is  not  feasible  to  test  each  item  in  the  lot.  Hence  a 
rejection,  whether  right  or  wrong,  is  practically  an  order  to  scrap  the  lot 
or  rework  it.  In  this  case,  the  cost  of  the  producer's  risk  is  painfully 
evident  especially  when  one  recalls  that  the  producer's  risk  causes  rejec¬ 
tion  of  acceptable  lots  which,  due  to  a  sampling  quirk,  give  the  false 
impression  of  being  rejectable.  In  any  case,  the  coat  of  rejecting  a  lot  is 
easy  to  calculate  and  it  is  given  in  the  following  symbolic  form;  (The 
meaning  of  the  symbols  is  provided  in  the  Glossary  appended  hereto. ) 

CR  =  (N  -  „)  (C„  -  Vg)  (Pp) 

It  should  be  obvious  that  C^,  the  cost  of  rejection,  can  be  computed  to 
the  last  penny;  very  few  approximations  are  necessary. 

COSTING  THE  CONSUMER'S  RISK.  It  is  otherwise  with  the  task  of 
calculating  the  cost  of  the  consumer's  risk  in  dollars  and  cents.  We  will 
recall  that  the  consumer's  risk  is  the  chance  he  takes  that  the  sample  may 
represent  an  unacceptable  lot  as  acceptable  material.  This  causes  him  to 
pay  for  and  take  possession  of  merchandise  which  contains  an  undesirably 
high  proportion  of  defective  material.  There  the  difficulty  begins;  to 
assess  the  cost  of  accepting  a  defective  lot  one  must  solve  the  problem  of 
fixing  the  cost  of  a  single  defective  item  and  follow  this  by  discovering  the 
actual,  percentage  of  defectives  in  the  lot.  If  the  latter  Information  were  at 
hand,  it  would  have  been  unnecessary  to  test  the  lot  for  acceptability  in  the 
first  instance  and,  had  the  teat  revealed  the  true  percent  defective  in  the 
lot,  it  would  nevor  have  been  accepted,  This  difficulty  pales  to  insignifi¬ 
cance  compared  with  the  problem  of  determining  the  cost  of  an  item  found 
to  be  defective  when  it  is  used.  This  is  particularly  true  of  exotic  items 
euch  as  space  rockets  and  military  material  where  failure  in  use  may  have 
strong  adverse  effect  on  national  prestige  and/or  security,  may  cause 
casualties  or  even  lead  to  tactical  defeat  in  situations  of  various  degrees 
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of  significance.  Almost  always  the  loss  due  to  the  defective  unit  depends 
upon  the  circumstances  surrounding  the  malfunction,  These  are  unpredict¬ 
able.  Thus,  a  piuxua'uurc  oheii  burn  may  cause  no  casualties  or  damage 
in  certain  situations  or  it  may  result  in  several  deaths  and  a  ruined  gun. 
Chance,  completely  unforeaeenable ,  will  determine  the  lose  in  each  case. 
Again,  how  can  we  compare  the  cost  of  a  dud  Jiand  grenade  on  the  practice 
field  with  the  loss  sustained  when  a  grenade,  thrown  into  an  enemy  machine 
gun  emplacement,  is  a  dud  and  the  brave  soldier  who  had  to  expose  himself 
to  the  gun  to  make  the  throw,  is  cut  down?  Someone  else  will  have  to  make 
that  throw  and  who  can  tell  how  many  casualties  will  be  sustained  to  silence 
the  gun  which  would  have  been  d  estroyed  had  the  grenade  functioned  in  the 
first  place?  The  additional  casualties  are  part  of  the  loss  associated  with 
the  dud.  How  can  anyone  predict  the  course  of  such  events?  If  one  wishes 
to  dramatize  this  problem  he  may  say  that  his  objective  is  to  put  a  price 
on  human  blood  and  look  into  his  crystal  ball  to  determine,  on  the  average, 
how  much  will  be  poured  out  on  each  defective  item. 

We  must  not  take  the  attitude  that  the  cost  of  the  beta  risk  can  never 
be  ascertained.  If  the  item  involved  is  a  component  and  the  defect  is  one 
that  will  be  caught  in  attempting  to  assemble  it  in  the  end  item  then  the 
nuisance  loss  of  this  type  of  defect  can  be  determined.  In  that  case,  the 
method  described  in  Reference  (1)  can  be  used  for  determining  sample  sise 
while  minimizing  the  total  cost  of  both  risks  and  of  sampling. 

As  we  shall  see  later,  the  coat  of  the  two  risks  strongly  Influence  the 
sample  size  determined  to  the  optimum  in  the  sense  of  reducing  the  total 
cost  to  a  minimum.  If  the  cost  assessed  therefore  is  very  high,  the 
optimum  sample  size  calculated  to  reduce  the  total  cost  to  a  minimum 
will  be  unrealistically  high  as  will  the  minimum  total  cost  computed  in 
these  circumstances.  In  a  democracy  such  as  ours,  great  value  is  placed 
on  human  life.  It  is  commonly  regarded  as  priceless  and  any  attempt  to 
set  a  monetary  value  on  blood  or  on  life  itself  is  considered  a  particularly 
obnoxious  form  of  sacrilege.  Yet  if  such  matters  are  to  enter  in  to  ths 
calculation  of  optimum  sample  size  in  a  specific  case,  a  monotary  value 
must  be  set.  The  engineer  seems  to  be  impaled  on  the  horns  of  an  insolu¬ 
ble  dilemma. 

HOW  TO  HANDLE  THE  CONSUMER'S  RISK.  Yet  a  solution  is  possible 
The  price  of  blood  or  life  must  simply  be  equated  to  zero.  In  other  words, 
It  must  be  eliminated  from  consideration  in  monetary  terms  as  suggested 
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in  Reference  (3).  Such  a  step  makes  the  problem  soluble,  In  this  caee,  the 
casualty-producing  defective  can  better  and  more  appropriately  be  handled 
by  prescribing  a  suitable  quality  level  for  acceptance,  To  adopt  this  . .  . 
course  is  equivalent  to  a  decision  to  eliminate  the  caeualty-producing 
defective  in  its  role  of  a  sample  size  determinant  and  to  direct  its  influence 
into  another  path,  so  that  it  will  act  to  determine  the  pertinent  quality  level 
instead, 

LOT  TOLERANCE.  One  way  to  handle  the  problem  of  determining 
the  optimum  sample  size  for  destructive  tests,  without  assessing  any  cost 
for  the  consumer's  risk  (this  is  the  same  as  ignoring  it  or  setting  it  equal 
to  zero)  is  provided  in  Reference  (2).  There  the  required  quality  level  is 
■  et  at  a  figure  appropriate  to  the  protection  desired  as  an  LTFD  (»  Lot 
Tolerance  Fraction  Defective;  see  Glossary)  which  is  a  level  of  quality'^,, 
poor  that  the  engineer  would  take  to  his  sick  bed  at  the  thought  of  having  to  ^ 
accept  consistently  material  of  LTFD  quality  though,  once  in  a  long  while, 
to  prevent  shutting  down  the  line  or  for  some  other  noble  purpose,  he  might 
be  willing  to  accept  such  a  lot.  By  setting  the  Consumer’s  risk  at  some 
low  figure  (e.  g.  0. 10  or  0.  05)  the  engineer  insures  that  only  one  lot  of  LTFD 
quality  out  of  10  or  20  submitted  will  be  accepted,  the  others  being  rejected. 
Obviously  no  producer  can  stand  the  economic  pressure  of  wholesale 
rejection,  so  the  quality  he  must  produce  to  stay  in  business  will  have  to 
be  a  good  deal  better  than  the  LTFD,  which  is  what  our  engineer  wants, 
Having  decided  on  a  proper  LTFD  the  paper  goes  on  to  show  how  the 
optimum  sampling  plan  is  computed  which  will  yield  the  desired  protection 
against  material  of  LTFD  quality. 

Reference  (1),  on  the  other  hand,  is  a  much  more  sophisticated  approach. 
However,  as  has  already  been  noted,  it  can  be  applied  only  where  the  cost 
of  the  beta  risk  can  be  computed  with  reasonable  correctness,  at  least  to 
the  extent  of  knowing  in  what  bail  park  the  doubleheader  will  be  played,  Our 
concern,  however,  is  with  the  area  within  which  the  cost  of  the  beta  risk 
cannot  be  approximated,  It  is  interesting  that  the  solution  herein  delineated 
can  be  used  equally  well  whether  one  can  or  cannot  eetimate  the  beta  riak 
cost  because  In  either  case  the  cost  can  be  ignored,  if  desired,  and  the 
acceptance  or  surveillance  quality  level  may  be  set  i.t  a  figure  which  will 
keep  the  outgoing  lot  percent  defective  at  aome  desired  limit  with  given 
probability  given  some  information  as  to  distribution  of  lot  quality.  That 
is,  wo  set  the  level  to  take  a  calculated  risk,  Then  we  figure  the  eampling 
plan  that  will  insure  that  outgoing  material  accepted  thereby  will  conform  to 
that  level  within  the  specified  risk, 
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Now  we  ahull  consider  how  this  purpose  maty  be  accomplished  by  the 
engineer  without  the  need  to  become  a  statistician,  amateur  or  profes- 
sional.  To  do  this,  we  propose  to  outline  the  procedure  "by  the  numbers" 
and  ask  the  engineer  to  accept  as  an  article  of  faith  that  the  procedure 
is,  in  fact,  valid  and  will  do  the  things  and  afford  the  protection  attributed 
to  it.  It  is  not  our  purpose  to  provide  mathematical  theory  or  proofs  here 
and  demand  that  you  grasp  them  before  we  will  permit  you  to  touch  the 
procedure.  Rather,  we  want  tc  present  a  method  which  you  can  grasp  in 
hands  grimy  from  contact  with  your  work  and  responsibilities  and  from 
a  knowledge  of  your  problem  and  needs,  proceed  to  calculate  a  sampling 
plan  tailored  to  do  what  you  want  it  to  do. 

COMPUTING  ACTUAL  COSTS.  If  we  consider  the  case  of  single 
sampling  (see  Glossary)  wherein  we  fix  the  consumer's  risk  (i.  e.  by 
establishing  some  desired  lot  tolerance  fraction  defective  with  a  10% 
chance  of  acceptance  -  the  consumer's  risk),  the  total  cost  of  the  inspec¬ 
tion  is  expressed  by  the  equation 

T  «  n  (Cy  +  CT)  +  (N  -  n)  (Pp)  (C^  -  Vg) 

Since  this  equation  is  basic  to  understanding  what  we  are  about  to  do, 
it  is  well  to  explain  it  without  going  to  the  Glossary,  T  is  the  total  cost 
of  testing  including  the  Producer's  Risk  the  cost  of  which  is  the  expression 
to  the  right  of  the  central  plus  sign.  To  the  left  of  that  sign  is  the  cost  of 
testing;  n,  the  sample  size,  times  the  sum  of  the  cost  of  one  unit  (which 
the  test  will  destroy)  and  the  cost  of  testing  it.  Thus,  if  the  sample  size 
is  35  and  we  shall  destroy  an  item  costing  $3  and  spend  $2  to  do  it,  then 
the  test  alone  will  cost  35  x  (3  ♦  2)  »  $175,  Now,  as  for  the  Producer's 
Risk,  the  rest  of  the  lot,  N  -  n,  is  subject  to  the  probability  (Pp)  that  it 

will  be  rejected  even  though  the  lot  is  really  acceptable.  The  symbol  Fp 

is  the  Producer's  Risk;  it  is  computed  as  1  -  Lp  by  subtracting  from 
unity  the  chance,  Lp,  that  a  lot  of  process  average  quality  (£),  presumably 
better  than  LTFD,  will  be  accepted.  If  unity  represents  all  possibilities 
and  Lp  is  calculated  as  a  decimal  fraction,  say,  0,  95  then  1  -  LJ5  is  the 
chance  of  rejection;  in  this  case  1  -  0,  95  »  0.  05,  Now  (N  -  n)  (Pp)  gives 

the  number,  on  the  average,  which  we  will  lose  from  the  lot  by  the  action 
of  the  Producer's  Risk,  We  may  not  lose  this  lot  but  when  we  do  lose  a 
lot  and  its  N  -  n  is  prorated  over  all  the  lots  we  do  not  lose,  each  lot  will 
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lost  about  (N  -  n)  (P_).  It  is  mains  onlv  to  cost  this  loss.  This  is  done  bv 

■  •  r 

multiplying (N  -  n)  (Pp)  bv  the  cost  of  one  item  less  its  salvage  value,  if 

any,  Cy  -  Vg,  If  an  item  costs  $10  and  can  be  reworked  for  $3,  then  Cy  -  Vg« 

$3  so  that  (C^j  -  Vg)  may  also  be  called  the  cost  of  reworking  the  item, 

When  the  appropriate  values  are  filled  in,  the  total  cost  T  of  using  any 
proposed  sampling  plan  against  material  of  the  quality  being  produced  (p) 
may  be  calculated,  A  bit  laborious  but,  as  you  can  see,  not  too  difficult. 

The  calculation,  from  scratch,  of  an  optimum  sample  sire  would  require 
quite  a  bit  of  work,  First,  as  indicated  in  (2),  one  would  have  to  determine 
a  succession  of  different  sample  sizes  and  an  associated  allowable  number 
of  defects  (c)  for  each.  Each  plan  must  be  designed  to  furnish  the  same 
protection  (same  Consumer's  Risk)  against  material  of  lot  tolerance  (LTFD) 
quality.  Then,  the  total  cost  of  each  plan  would  be  computed,  using  the 
above  equation,  It  would  require  facility  in  using  a  table  of  probabilities. 
While  this  would  not  be  difficult  to  learn,  such  a  table  is,  after  all,  a 
statistician's  reference,  Happily,  Ellner  and  Savage  (4)  have  developed 
short-cut  methods  for  calculating  optimum  single  and  double  (see  Glossary) 
sampling  plans  utilizing  graphical  methods  and  graphs  developed  by  Dodge 
and  Romig  (5),  These  graphs  are  reproduced  and  appended  hereto  with 
the  kind  permission  of  the  originators  and  publishers  and,  in  any  case,  can 
be  consulted  in  (5). 

THE  WORK  OF  DODGE  AND  ROMIG,  It  is  generally  acknowledge  that 
Dodge  and  Romig  are  the  fathers  of  statistical  sampling  as  used  in  quality 
assurance  work.  It  is  astonishing  to  see  how  sophisticated  their  thinking 
was,  even  in  its  earliest  published  form  in  the  Bell  Telephone  Technical 
Journal.  Their  methods  are  intensely  practical  but  that  should  not  sur¬ 
prise  anyone  since  they  were  engineers  faced  with  the  eminently  practical 
problem  of  sampling,  While  their  rejected  lots  would  be  inspected  100%, 
they  recognised  that  the  cost  of  such  100%  inspection  is  an  economic  loss, 
Their  sampling  plans  were  calculated  to  minimize  the  over-all  cost  of  the 
inspection  operation  incl  .ding  the  100%  inspections  caused  by  the  Producer's 
Risk.  Therefore  the  idea  of  optimizing  sample  size  for  minimum  coat 
originated  with  Dodge  and  Romig.  The  use  of  the  same  principle  for 
destructive  or  expensive  testing  where  100%  inspection  of  rejected  lots  was 
patently  Impracticable  waz  urged  by  (2)  and  (4),  substituting  C^T  -  Vg  for 

the  Dodge  and  Romig's  100%  inspection  of  rejected  lots,  With  this  great 
■imilarity  in  basic  ideas,  it  is  not  too  surprising  that  we  can  use  Dodge 
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and  Komig : s  graphical  methods  to  avoid  ?.  d*al  of  computational  work 

which  might  be  not  only  laborious  but  confusing  to  the  non- statistician.  To 
avoid  the  latter,  we  propose  to  develop  single  and  double  sampling  plans 
using  the  Dodge-Romig  graphs  and  to  proceed  step  by  step  explaining  only 
as  required  to  facilitate  achievement  of  the  final  objective  •  the  sampling 
plan, 

CONTROLLING  THE  PROCESS,  In  their  eagerness  to  insure  receipt 
of  high  quality  material,  engineers  can  easily  fall  into  the  trap  of  specify¬ 
ing  acceptance  criteria  so  high  as  to  increase  production  and  inspection 
costs  beyond  reason  and  hamper  production  of  a  smooth  flow  of  acceptable 
material.  For  the  dubious  advantage  of  an  exceedingly  low  outgoing 
proportion  of  defective  material,  the  consumer  pays  through  the  nose. 

There  are  other  ways  to  do  this  without  incurring  prohibitive  costs  and 
strangling  production,  Perhaps  the  most  effective  way  is  to  engineer 
production  and  establish  effective  quality  controls  at  the  right  points  on 
the  production  line  so  that  production  of  the  most  critical  or  significant 
types  of  defects  will  be  almost  Impossible,  Another  way,  not  as  effective 
and  more  costly,  but  easier  and  more  convenient  for  the  purchaser  is  to 
establish  an  LTFD  at  such  a  level  that,  to  avoid  a  costly  high  proportion 
of  rejections,  the  producer  will  have  to  maintain  an  average  quality  output 
well  above  the  LTFD, 

ESTABLISHING  THE  LTFD.  In  establishing  the  LTFD  we  shall  assume 
a  Consumer's  Risk  of  10%  or  0^10  for  two  reasons,  First,  ever  since  Dodge 
and  Romig  first  calculated  their  tables  this  has  been  the  risk  conventionally 
accepted  for  the  LTFD,  Second,  their  graphs  are  based  on  an  0, 10  risk, 

The  engineer  should  set  his  LTFD  at  some  fraction  defective  such  that, 
even  if  a  lot  of  LTFD  quality  were  accepted  on  rare  occasion,  it  would 
causa  no  insurmountable  problem  in  the  field.  Since  sampling  plans  devel¬ 
oped  by  our  method  with  reject  lots  of  LTFD  quality  nine  times  out  of  tan, 
if  the  contractor  would  regularly  produce  material  of  this  quality  he  would 
surely  face  economic  disaster,  If  the  supplier's  Producer's  Risk  is  to  be 
at  a  tolerable  level  he  must  produce  material  by  a  process  which  is  statisti¬ 
cally  controlled  to  give  a  process  average  (?)  proportion  defective  very 
roughly  1/ 3  or  l/4  of  the  LTFD,  Thus,  if  the  LTFD  is  0,  08,  the  supplier 
should  produce  a  f>  of  about  0,  02  or  0.  03  to  avoid  excessive  loss  due  to  the 
Producer's  Risk,  If  the  supplier's  {5  is  much  lower  than  the  LTFD  the 
optimum  sample  sise  will  be  relatively  low, 
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The  engineer  should  choose  an  LTFD  that  will  give  him  what  he  needs 
at  an  acceptable  price.  From  the  facts  already  indicated,  he  must  have 
a  reasonable  expectation  that  the  supplier  will  be  able  to  produce  a  con¬ 
trolled  p  which  is  l/3  LTFD.  If  he  cannot,  his  prices  will  have  to  be 
raised  to  cover  the  excessive  rejections  he  is  sure  to  experience.  The 
engineer  must  avoid  demanding  material  of  prohibitively  high  quality 
solely  for  the  purpose  of  bolstering  his  reputation  for  designing  items 
which  work  all  the  time.  He  must  remember  that,  if  the  supplier  is  try¬ 
ing  to  make  material  at  a  controlled  p  =  1/ 3  LTFD,  very  rarely  will  the 
process  make  a  lot  of  LTFD  quality  and,  even  if  it  does,  the  chance  of  its 
being  accepted  is  only  one  in  ten,  so  the  engineer  can  rest  assured  that, 
for  practical  purposes,  almost  all  accepted  lots  will  be  much  better  than 
LTFD  quality.  With  this  in  mind,  he  can  afford  to  be  fairly  generous  in 
setting  the  LTFD. 

Perhaps  as  good  a  way  as  any  is  to  assume  some  realistic  p  which 
the  engineer  feels  a  qualified  supplier  can  maintain  under  statistical  control 
when  producing  the  item  in  question.  Then  the  engineer  multiplies  p  by 
4  and  3  and  asks  whether  a  product  of  quality  4p  or  3p  can,  on  rare 
occasion,  be  accepted  without  causing  excessive  trouble  to  the  user. 

Using  this  as  a  criterion  he  sets  his  LTFD  at  4p  if  possible,  at  3p  other¬ 
wise.  The  engineer  should  realize  that,  if  the  supplier  maintains  control 
over  his  quality  a  lot  of  LTFD  quality  will  almost  never  be  produced, 
much  less  accepted.  The  supplier  should  recognize  that  if  a  sampling 
plan  is  computed  on  an  LTFD  basis  he  would  be  well  advised  to  get  his  proc¬ 
ess  under  statistical  control  at  a  p  no  greater  than  l/ 3  LTFD  and  keep  it 
there.  If,  for  some  reason,  the  LTFD  must  be  set  at  some  figure  notice¬ 
ably  less  than  3p,  the  engineer  should  expect  higher  prices,  uncertain 
deliveries  or  repeated  requests  for  waivers  or  changes  in  contract  require¬ 
ments.  The  supplier  can  anticipate  occasional,  even  frequent  rejections 
and  organize  with  this  possibility  in  mind.  The  above  procedure  is  only  a 
useful  rule-of-thumb.  By  making  a  number  of  trial  calculations,  the 
engineer  can  satisfy  himself  that  when  p  is  very  small  compared  with  LTFD, 
the  sample  size  required  will  be  relatively  small  and  rejections  will  be 
few.  As  p  approaches  the  LTFD,  sample  size  will  be  at  a  high  and  rejec¬ 
tion  will  tend  to  occur  in  9  cases  out  of  10. 

DESIGNING  THE  OPTIMUM  SINGLE  PLAN /EXAMPLE  1).  To  illus¬ 
trate  how  to  design  a  single  sampling  plan,  we  shall  use  the  example 
furnished  in  (4).  First  we  shall  list  by  symbols  the  things  we  need  to  know 
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quantitatively,  If  any  of  this  information  is  lacking,  it  is  advisable  to 
use  yum  best  guss;  -~d  make  any  cor’,»<'Hnn  which  later  information 
indicates  to  be  suitable. 

N  »  5000  Cy  »  85 

LTFD  «  pt  a  0,  07  CT  ■  SL0 

p  *  0.  02  Vg  =  $3 

We  calculate  the  qualtities  A  -  C  +  C_  =  815  and  B  =  C  -  Vc  =  $2. 

u  i  US 

Usually  A  and  B  can  be  determined  quite  accurately  but  they  are  not  as 
important  as  the  ratio  .£•  Using  these  figures,  we  calculate  the  follow* 
ing:  A 


p  N  «  0, 07  x  5000  *350 

2 

-g-  ■  approximate  equivalent  lot  siae  ■  x  5000  ■  667 


"  or  "  °'29, *nd 


B 


N 


pt  X  *  (aPPIpoximat®  equivalent  lot  sise)  «  0, 02  x  667  =  46.  7, 


We  enter  Figure  2  with  p 


t  A 


■  46.  7  and 


£. 

P. 


0,  29  and  get  an  acceptance 


number  c  ■  4.  Now  going  to  Figure  3,  we  follow  the  curve  for  an  acceptance 

number  of  4  and  we  find  it  leaves  the  chart  at  p  N  of  200.  Our  p  N  is  350 

»  * 

but  since  the  curves  for  c  ■  0  to  c  ■  10  remain  parallel  to  the  horiaontal 
axis  past  p  N  ■  200,  we  read  (p  )  (sample  siae)  or  p  n  ■  8,  Since  p  ■  0,  07 

t  g  I  t  t 

we  find  n  «=  g—gg  ■  114.  We  substitute  114  in  the  expression  for  the  exact 

equivalent  lot  siae,  p  [-p^  +  (1  -  -r-)n]  which  converts  to  0,Q?[tt  x  5000  + 

2  *  A  A  1 5 

(1  -  ^-)114]  *  53.  6.  We  could  not  calculate  the  exact  equivalent  lot  siae 
before  this  because  we  need  to  know  n,  the  sample  siae.  That  we  obtained 
by  first  using  the  approximate  equivalent  lot  siae,  We  re-enter  Figure  2 
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with  the  new  mHmaf*  nf  n  f 1  At  Tt  S3  A  a«.,4  £.  —  h  9a  x>J 

4  £  %  ~  » - -  -  - - - f  —  —  - - p  “*•« 

get  c  =  5.  Now  we  re-enter  Figure  3  with  c  »  5  and  p^N  =  3&  and  read  9.  2 


(by  using  dividers  and  a  scale).  Since  pt  =  0.  07,  0.  0?n  =  9.2  whence 
n  =  131.  The  optimum  single  sampling  plan,  then,  ii  n  ■  131,  c  *  5.  We 
can  check  this  by  recalculating  the  long  exp  res*  ion  above  and  getting  54,5 
which  when  used  to  enter  Figure  2  again  with  a  0.29,  finds  c  ■  5  un¬ 
changed,  That  is  all  there  is  to  it. 


INFLUENCE  OF  THE  PROCESS  AVERAGE,  p.  To  insure  that  the 
optimum  in  sampling  economy  is  maintained,  the  process  average  should 
be  recomputed  every  5  or  10  lots,  If  any  siseable  change  is  noted,  it  would 
be  wise  to  recompute  the  sampling  plan,  which  is  not  an  onerous  task  as 
you  have  seen.  The  question  may  be  put  as  to  what  value  to  use  for  p  when 
calculating  the  original  sampling  plan,  when  no  quality  history  exists  for 
the  production  line,  At  such  a  time,  your  best  guess,  as  to  the  average 
quality  the  line  is  expected  to  produce  is  adequate  or  you  may  prefer  to 
estimate  $  conservatively  at  about  0.  3p^.  It  probably  will  not  make  too 

much  difference  either  way  since,  even  if  the  estimate  is  off  somewhat,  it 
will  not  be  too  far  away  and  will  be  changed  as  soon  as  a  quality  histroy 
becomes  available.  As  an  exercise,  one  might  vary  the  process  average, 
using  some  figures  much  higher  and  much  lower  than  p  ■  0,  02  and  notice 
the  effect  on  the  sample  sUe  which  results  from  the  change. 

DOUBLE  SAMPLING.  Some  time  ago,  double  sampling  and  the  related 
multiple  sampling  were  regarded  as  ways  to  reduce  the  over-all  cost  of 
sampling  since,  for  sampling  plans  giving  the  same  protection  the  total 
number  of  sample  items  needed  for  single  sampling  was  normally  noticeably 
more  than  what  double  sampling  demanded  which,  in  turn,  was  greater  than 
what  multiple  sampling  required.  Thus,  if  the  amount  of  retesting  could 
be  kept  down,  as  when  quality  is  either  very  good  or  very  poor,  appreciable 
savings  appear  possible.  Since  the  system  for  calculating  optimum  single 
sample  plans  takes  into  account  changes  in  sample  slaes  when  p  changes, 
it  possesses  some  of  the  advantages  of  double  and  multiple  sampling  without 
the  disadvantages.  Again,  many  like  the  idea  of  getting  a  second  chance 
with  double  sampling,  several  chances  with  multiple  sampling.  One  does 
not  feel  so  tied  down  to  the  one  chance  of  the  single  sample.  This  is,  of 
course,  purely  psychological  for,  mathematically,  there  is  a  price  to  pay. 
Additional  costs  must  be  borne  in  selecting  second  and  other  samples  that 
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are  used  only  infrequently.  There  ia  the  physical  burden  and  inconvenience 
of  handling  rr.czz  sompls  itcius  and  of  returning  unuaed  eamploa  to  the 
parent  lots.  Then,  too,  when  retests  become  more  frequent  than  originally 
anticipated,  heavy  work  loads  are  experienced  leading  to  over-work,  fatigue 
and,  eventually,  to  error.  These  factors  have  caused  double  and  multiple 
sampling  to  lose  some  of  their  popularity  and  led  to  greater  dependence 
upon  and  use  of  single  sampling  plans.  Nevertheless,  we  shall  include  a 
method  for  computing  optimum  double  sampling  plans. 

DESIGNING  THE  OPTIMUM  DOUBLE  SAMPLING  PLAN  (EXAMPLE  2). 
For  this  example  we  shall  use  the  figures  used  in  Example  1.  To  spare  you 
the  trouble  of  looking  them  up  they  are  listed  below: 

N  *  5000  Cy  *  $15 

LTFD  a  p  ■  0,  07  CT  ■  810 

f>  m  0.  02  .  Vg  *  83 

Again  we  calculate  A  *  *  $15  and  B  ■  •  Vg  =  $2,  Using  these 

figures  we  calculate 


p  N  -  0.  07  x  5000  ■  350 
z 


a  approximate  equivalent  lot  size  * 


2_ 

15 


x  5000  ■  667 


2-  ■  c  o.  286  and 
Pt  .  07 

Pt  B^/A  *  (p^)  (approximate  equivalent  lot  sice)  ®  0.  02  x  667  ■  46,  7. 

To  determine  the  respective  c  numbers  for  our  double  sampling  plan  we  use 
Fig  2-7  which  is  analagous  to  Fig  1-2.  We  enter  Fig  2-7  with  p^  B^/A  =  46.  7 

for  the  ordinate  or  vertical  component  and  p/p{  =  0,  286  for  the  horizontal 

component  or  abscissa,  We  find  a  1  and  s  7,  almost  inside  *  8. 
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Now  we  use  Fig  2-8  and,  at  p^N  =  350,  the  curve  for  *  1  gives  a  reading 

of  4.  5  on  the  ordinate  which  represents  p  n  or  p  times  the  first  sample 

*  4  5  * 

sire,  Since  p^n^  =4.5  and  =  0,  07,  n^  -  q- =  64,  Similarly  we  look 
UP  c2  P^N  =  350  and  we  find  an  ordinate  of  12,  8  which  now  represents 
Pt  (»  +  n2).  Now  if  pt  (nx  +  n2)  -  12,  8  and  pt  =  0.  07  then  ^  +  n2  =  *  1*3 

Since  n^  =  64,  n2  =  183  -  64  *  119.  As  before,  this  is  a  firet  approximation 
to  the  sampling  plan  we  want.  Substituting  in  the  expression 


+  (1  -  |)  (nx  +  n2) 


we  get  667  +  (1  -  2/15)  (183)  =  826.  Back  we  go  to  Fig  2-7,  using  p  (826)  ■ 

0.  07  x  826  =  57.8  and  we  get  c1  =  1  and  c2  ■  8.  Again  we  enter  Fig  2-8 
with  ptN  =  350  as  the  abscissa  and  for  =  1  we  get  p^  =4.5  so  that  n^  =  64 
as  before.  However  for  c2  =  8,  w a  get  pt  (n^  +  n2)  =  14.  0  whence  n1+n2  * 
14.0 

0 1  Q7  a  200,  from  which  »  200  -  64  =  136,  The  sampling  plan  then  is 

C1  =  C2  5  ni  *  ^4,  n2  =  136.  ^  desired,  the  sample  sizes  can  be 

rounded  to  n^  =  65,  n2  =  135  without  too  great  a  change  in  the  effect  of  the 

plan.  As  you  can  see,  the  calculations  are  a  bit  more  involved  for  the 
double  sampling  plan  as  compared  with  the  single  sampling  plan  but  the 
principle  la  the  same. 

The  desire  to  keep  the  presentation  simple  requires  omission  of 
several  facets  which  might  be  useful  such  as  an  easy  way  to  calculate  the 
expected  total  coet  of  a  given  sampling  plan  if  p  is  known.  However,  if 
this  information  is  required  it  can  be  obtained  from  other  graphs  in  (5). 

In  all  the  previous  discussion,  it  was  assumed  that  the  only  informa¬ 
tion  available  regarding  the  quality  of  the  lot  to  be  tested  was  that 
developed  from  the  Bample.  In  an  actual  production  situation  a  substan¬ 
tial  amount  of  engineering  information  is  developed  during  the  production 
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cycle  which,  properly  interpreted,  can  indicate  whether  the  process  is  in 
statistical  control  and,  therefore,  may  be  considered  to  be  producing  sub* 
stantially  homogeneous  material.  If  the  material  is  homogeneous  from  lot 
to  lot  thsn  the  results  of  teste  generated  in  previous  lots  may  be  con¬ 
sidered  to  have  eignificant  bearing  on  the  results  expected  in  the  latest 
lot.  Hence  when  statistical  control  hat  been  established,  the  o ample  size, 
lot  by  lot,  can  be  reduced  substantially  and  remain  reduced  provided  no 
evidence  is  obtained  indicating  loss  of  control. 

Basically,  if  advantage  is  taken  of  available  engineering  knowledge 
of  previous  experience  with  the  process  sampling,  testing,  and  their 
attendant  coats  may  be  reduced.  This  notion  lend*  itself  readily  to  statis¬ 
tical  ingenuity  but  the  engineer  will  require  the  assistance  of  a  statistician 
to  take  advantage  of  thj  possibilities,  A  number  of  ingenious  scnemes  to 
permit  useful  employment  of  existing  engineering  data  can  be  devised  to 
reduce  the  sample  size  and  test  costs  below  the  "optimum"  solution  just 
described, 

The  author  desires  to  express  his  appreciation  and  gratitude  to  Ellnrx 
and  Savage  for  permission  to  uSe  the  results  of  thoir  research  and  most 
particularly  to  Professor  Harold  F.  Dodge,  Dr.  Harry  G.  Romig,  and 
John  Wiley  and  Sons,  Inc.  for  their  unselfish  generosity  in  allowing 
reprinting  of  their  graphs  without  which  this  work  would  have  been  impossible 
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Cp  a  Cost  of  rejection 
N  a  Lot  sige 
n  «  Sample  size 
Cy  *  Cost  of  a  single  unit 

Vg  »  Salvage  value  of  a  single  unit  or  its  value  as  rework  material 

'  Producer's  risk:  probability  (expressed  as  a  decimal  frac¬ 
tion)  that  the  sample  will,  on  test,  represent  the  lot  to  be 
unacceptable  when  it  is,  in  fact,  quite  acceptable 

Cg  3  Cost  of  sample  item 

CT  ■  Cost  of  testing  a  oingle  unit 

A*CU+CT  «  The  cost  of  destroying  one  item  in  testing 

B  =  Cy  -  Vg  =  The  value  of  one  rejected  item 

c  ■  Acceptance  number,  the  maximum  number  oi  defectives  that  will 
be  permitted  in  a  sample  of  size  n  from  an  acceptable  lot. 

If  more  than  c  defectives  tire  observed  in  the  sample  of  n 
items  the  lot  will  be  rejected. 
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nm TUT  TT  C  A  \  inr  TKT^  PvwnAt  o 

-  *-*-  «•**’**  W  1  iUUWMU 

=  Size  of  first  sample 
=  Size  of  second  sample 

nl  +  n2  "  ®*ZB  coml3^rie<1  first  and  second  samples 

cj  3  Acceptance  number  for  first  sample,  n^.  If  or  fewer  defer* 
tives  are  found  in  n^,  the  lot  is  accepted  straight-away,  If 
the  number  of  defectives  found  in  n^  is  greater  than  c^  but 
equal  to  or  less  than  cgithe  second  sample,  n2,  is  tested 
and  the  number  of  defectives  in  nj  and  in  n2  is  totalled,  If 
that  number  is  greater  than  c2  (the  number  of  defectives  per¬ 
mitted  in  nj  +  n2)  the  lot  is  rejected.  If  c2  or  less  defectives 
are  found  in  n1  +  n2  on  retest,  the  lot  is  accepted, 


\  DEFINITIONS 

\ 

Single  Sampling  -  A  system  of  sampling  whereby  a  single  sample  is 
drawn  from  a  lot.  and  the  acceptability  of  the  lot  is  determined  from  the 
results  obtained  in  testing  the  sample.  No  retest  is  permitted  if  results 
are  unfavorable, 

Process  average  («',  -  The  apparent  proportion  of  percent  of  defec¬ 
tives  manufactured  by  the  production  process,  It  is  generally  computed 
by  dividing  the  total  number  of  defectives  found  in  the  samples  taken  from 
the  last  few  lots  tested  (5  or  10)  by  the  sum  of  the  sample  sizes.  This 
gives  j5  as  a  decimal  fraction, 

Lot  Tolerance  Fraction  Defective  (LTFD  or  0-)  -  Lot  quality,  expressed 
as  a  decimal  fraction  defective,  sc  poor  that  we  want  to  permit  only  a  small 
chance  or  probability  (the  Consumer's  Risk,  say  one  chance  in  10  ■  10%  = 
0.10  probability)  that  the  sampling  plan  will  permit  acceptance  if  such  a  lot 
is  submitted, 

Double  Sampling  -  A  system  of  sampling  wherein  two  samples  are 
taken  and  one  se+  of  acceptance  and  rejection  criteria  are  furnished  for  each 
sample.  If  the  results  obtained  in  testing  the  first  sample  meet  neither  the 
acceptance  nor  the  rejection  criterion  for  that  sample,  the  second  sample  is 
tested  (called  the  retest)  and  the  decision  is  made  using  the  second  set  of 
criteria.  A  decision  is  always  possible  using  the  second  set  of  criteria 
after  the  retest. 
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*  CHART  FOR  FINDING  ACCEPTANCE  NUMBER  OF  SINGLE 
SAMPLING  PLAN.  (CONSUMER'S  RISK,  0.10). 


F«G.  3.* 


I  M  45  7  rO  20  30  50  70  500  200 

(LOT  SIZE) 

CURVES  FOR  FINDING  SIZE  OF  SINGLE  SAMPLING  PLAN. 
(CONSUMER’S  RISK,  0.10). 


‘Reproduced  in  part,  by  permission,  from  "Sampling  Inspection  Tables” 
by  Dodge  &  Romig,  published  by  John  Wiley  &  Sons,  Inc. 


|pt]  ("EQUIVALENT”  LOT  SIZE) 


FIG.  4/  CURVES  FOR  FINDING  THE  MINIMUM  COST  OF  INSPECTION  PER  LOT 
(SINGLE  SAMPLING  PLAN  -  CONSUMER'S  RISK,  0.10) 


•Reproduced  In  part,  by  permission,  from  ’’Sampling  Inspection  Tables” 
by  Dodge  S  Romig,  published  by  John  Wiley  &  Sons,  Inc. 
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PROCEDURES  FOR  FINDING  TOTAL  SAMPLE  STATISTICS 
FROM  SUBSAMPLE  STATISTICS 

Paul  C .  Cnv 

Reliability  and  Statistic*  Division 
Army  Missile  Test  and  Evaluation  Directorate 
White  Sands  Missile  Range.  New  Mexico 

ABSTRACT,  While  procedures  for  obtaining  the  variance  for  a  total 
■ample  from  subsample  statistics  is  fairly  well  known,  there  appear  to  be 
very  few  instances  in  which  such  procedures  are  found  in  print.  Therefore, 
twenty-five  formulas  are  presented  which  are  in  one  way  or  another,  related 
to  obtaining  the  mean  and  variance  for  a  total  sample  from  subsample 
statistics.  In  addition,  techniques  are  demonstrated  for  using  these  formulas 
to  determine  the  mean  and  variance  for  a  sample  in  which  a  portion  of  the 
observations  have  been  modified,  some  have  been  added,  or  a  few  have 
been  deleted. 

The  discussion  includes;  applications  of  these  formulas;  precautions 
which  should  be  observed;  methods  for  deriving  the  formulas;  and,  procedure* 
for  their  use. 

1.  INTRODUCTION.  This  report  presents  techniques  and  formulas  for 
determining  the  mean  and  variance  of  a  total  sample  if  this  sample  has  been 
partitioned  into  a  set  of  non  overlapping  and  mutually  exhaustive  subsamples; 
and  tha  mean,  variance,  and  sample  sixe  are  known  for  each  subsample. 

Similarly,  techniques  are  discussed  for  changing  the  variance  when 
observations  are  added  to,  deleted  from,  or  changed  in  a  sample.  Procedures 
for  deriving  these  formulas  are  discussed  and  some  of  the  derivations  are 
included  in  this  report. 

Most  people  know  these  formulas  exist,  and  they  are  not,  for  the  most 
part,  difficult  to  derive.  However,  they  are  often  useful  and  it  is  usually 
difficult  to  find  them  in  print;.  To  illustrate  this  point,  a  total  of  eighty- six 
statistics,  design  of  experiments,  probability,  sampling,  and  quality  control 
texts  were  reviewed  and  of  that  number,  only  two*  included  a  diecuseion 


,!<(l)  Sampling  Inspection  by  Variables,  Bowker  and  Goode,  pp.  62,  63,  and 
92. 

(2)  Techniques  of  Statistical  Analysis,  Eieenhart,  Haetay,  and  Wallie, 
pp.  42-43. 
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on  how  to  determine  the  total  vari&nce  from  subsample  statistics.  From 
this,  it  apn*«r*  fha.t  while  the  formula*  and  procedures  wmch  are  presented 
here  may  be  well  known,  few  authors  seem  to  have  bothered  to  put  them 
in  print.  Forthsrmors,  it  has  been  observed  that  many  people  have  needed 
certain  of  these  formulas  and  not  being  able  to  locate  them  in  print  have 
found  it  necessary  either  to  spend  considerable  time  deriving  them  or  eim- 
ply  to  do  without. 

One  obvious  method  for  obtaining  the  mean  and  variance  for  a  total 
sample  is  to  gather  the  raw  data  from  all  the  eubs&mples  and  compute  these 
statistics  by  conventional  procedures.  It  is  equally  clear  that  use  of  raw 
data  will  be  unsatisfactory  if  the  subaamples  are  quite  large  because  of  the 
amount  of  work  involved;  and  the  raw  data  certainly  cannot  be  used  in  those 
frequent  cases  in  which  it  is  no  longer  available. 

II.  APPLICATIONS  AND  PRECAUTIONS.  The  following  are  usee  of  the 
procedures  and  formulas  of  this  section: 

A.  After  estimating  the  mean  and  variance  for  a  number  of  different 
populations,  a  research  worker  may  want  to  know  the  mean  and  variance  for 
a  population  composed  of  a  combination  of  these  populations.  This  would 

be  accomplished  by  combining  the  samples  from  the  sub -populations  to 
obtain  a  total  sample. 

(1)  An  example  of  this  would  be  the  case  of  production  lotu.  The 
mean  and  variance  will  be  known  for  a  sample  from  esch  lot,  but  an  estimate 
of  the  mean  and  variance  for  the  entire  production  may  be  desired.  To  obtain 
thisit  would  be  necessary  to  combine  the  lot  samples  to  obtain  a  total  sample. 

(2)  A  second  example:  After  conducting  an  analysis  of  variance  to 
determine  the  effect  of  certain  treatments,  the  research  worker  may  want  to 
estimate  the  mean  and  variance  for  a  population  composed  of  several  sub¬ 
populations,  each  identified  by  a  certain  treatment  level,  For  this,  sub¬ 
samples  could  be  combined  to  form  a  total  population, 

B.  Sample  data  may  come  from  many  sources,  for  example,  from 
several  parts  of  the  country,  from  several  agencies,  or  from  several  periods 
of  time,  and  it  may  frequently  bs  desirable  to  combine  the  data  to  form  one 
total  samplo.  Obviouely,  it  may  be  that  only  the  mean,  variance  and  sample 
else  for  each  subsample  are  available  or  can  easily  be  transmitted  rather 
than  the 'complete  raw  data. 
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C,  Frequently,  sample  data  has  been  completely  analyzed  when  it 
becomes  evident  that  a  few  observations  must  be  added,  certain  observa¬ 
tions  should  be  deleted,  ur  a  lew  should  be  corrected.  The  procedures  of 
this  report  may  be  very  useful  in  changing  or  correcting  the  original  esti¬ 
mates  of  the  mean  and  variance  as  a  result  of  changing  or  correcting  the 
basic  data, 


In  this  connection,  these  formulas  may  be  useful  in  computing 
statistics  associated  with  moving  averages. 


D.  As  a  final  application,  those  who  teach  statistics  at  the  Sophomore 
or  Junior  level  might  find  the  derivation  and  application  of  some  of  these 
formulas  an  interesting  assignment. 


The  main  precaution  to  observe  when  using  these  formulas  Is  that  the 
total  sample  may  represent  a  population  with  such  strange  or  unknown  charac¬ 
teristics  that  an  estimate  of  the  variance  would  be  useless  when  obtained. 

For  example,  a  total  population  composed  of  k  normal  sub -population s*  each 
with  a  different  mean  and  variance,  is  not  likely  to  be  normal  or  even  close 
to  normal. 

On  the  other  hand,  It  is  quite  posaible  that  the  characteristic ■  of  tha 
total  population  will  be  known  and  the  estimates  of  its  parameters  useable. 

For  example,  the  sub-populations  may  not  be  normal,  but  it  may  be  possible 
to  combine  them  to  form  a  normal  total  population,  Similarly,  the  variance 
for  the  total  population  may  be  needed  to  describe  the  distribution  of  sample 
mr.ans,  and  this  distribution  should  approach  normality  regardless  of  the 
distribution  of  the  total  population. 

Another  precaution  is  that  one  should  observe  whether  the  ratio  of  each 
suosample  size  to  the  total  sample  size  is  about  the  same  as  the  ratio  of  the 
corresponding  sub-population.  If  this  is  not  the  case,  weighting  factors 
should  be  Introduced  to  obtain  the  correct  ratios, 

As  a  final  precaution,  before  combining  cubsamples  to  form  a  total 
sample,  one  should  always  observe  whether  it  is  inherently  reasonable  to 
combine  such  data.  That  is  to  say,  the  subsamples  may  contain  such  differ- 
enr.  types  of  observations  that  combining  them  would  be  nonsense. 
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Th-  actual  differ tmcst  between  &  total  eitimate  and  a  pooled  estimate 
of  the  variance  should  be  discussed  at  this  point. 

A  total  variance  is  the  variance  of  one  complete  sample,  which  has  been 
broken  down  into  two  or  more  subsamples.  No  assumptions  are  made  con¬ 
cerning  the  populations  corresponding  to  each  subsample.  More  specifically, 
no  assumption  is  made  concerning  the  variances  of  these  populations,  How¬ 
ever,  it  is  assumed  that  when  the  total  variance  has  been  obtained,  its 
corresponding  total  sample  corresponds  to  a  population  with  known  charac¬ 
teristics.  If  this  were  not  so,  there  would  be  little  purpose  :'in  a  total  variance. 

The  pooled  estimate  of  the  variance  can  be  obtained  from  subsample 
statistics,  just  as  a  total  variance,  It  differs,  however,  in  that  it  is  in  no 
way  related  to  a  total  sample  or  a  total  population.  Therefore,  no  assump¬ 
tions  need  be  made  concerning  a  total  population.  The  assumption  Is  made, 
however,  that  all  subsamples  come  from  the  same  population,  or  at  least  * 
from  populations  which  have  equal  variances.  The  pooled  estimate  is  then 
an  improvement  over  each  of  the  estimates  obtained  from  any  single  sub¬ 
sample. 

III.  DEFINITIONS. 

A.  k  a  Number  of  subsamplos, 

B.  n^  =  Size  of  the  i**1  subsample  (i  ■  I,  2,  .  ,  ,  k), 

(If  all  n,  are  equal,  use  n) 

C.  N  *  Size  of  the  complete  sample, 

(1)  N  »  (i  »  1,  2,  ...  k). 

(2)  N  ■  kn  if  .all  n,  are  equal, 

til 

D.  »  Mean  for  the  i  subsample, 

E.  8  =  Mean  for  the  total  sample. 

2 

F.  s  «  Variance  for  the  complete  sample, 

s  =  Standard  deviation  for  the  total  sample. 
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G.  s, 


Variance  for  the  ik“  subsample. 


H.  s  U  Pooled  estimate  of  the  variance. 
P 

IV.  PROCEDURES. 

A.  The  overall  mean  x; 


(1)  If  the  n.  are  unequal: 


*. 

jf. 

f 


% 

if- 

i 


a 

1 


=  £rV*i 
x=  “n“ 

(2)  If  the  n^  are  equal: 

nEi,  ;  Ex. 

£  =  _ L  =  _ * 

N  k 


Formula  (I) 


(IX) 


B.  Pooled  Estimate  of  the  Variance  a 

P 

The  pooled  estimate  of  the  variance  is  actually  an  average  of  the 
subsample  variances,  and  should  be  computed  only  if  there  is  reasona  e 
assurance  that  all  subsamples  were  selected  from  populations  with  equal 
variances. 

(l)  If  the  n  are  unequal; 

1  2 


•  ih 


(2)  If  the  n^  are  all  equal; 


2  (n-l)E  •i 
‘p  *  FTk 


Es. 


(Ill) 


(Ilia) 


C.  Determining  the  Variance  from  an  Analysis  of  Variance  Table. 
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One  metVirvri  for  computing  both  the  total  variance  and  the  pooled 
estimate  ox  variance  is  by  preparing  a  single  variable  analysis  of  variance 
table.  In  addition  to  determining  the  variance b,  it  will  also  be  possible 
to  test  the  null  hypothesis  of  the  equality  of  subsample  means.  This  is 
described  by  Table  1. 


TABLE  1  •  The  Analysis  of  Variance  Method 


Degrees  of 
Freedom 

Sum  of 
Squares 

Mean 

Square 

B 

T  reatments 

k-1 

TR 

tr 

Error 

N-k 

E 

•p2 

Totsd 

N-l 

T 

s2 

— i 

Table  1  is  completed  as  follows; 

(1)  Complete  all  entries  under  degrees  of  freedom. 

(2)  Compute  and  enter: 

2  2 
E  >*  X^-lJe,  ,  or  if  all  are  equal  E  ■  (n-l)X  s^  . 

(3)  TR  *  X  ^  Qr  ^  are  equal  TR  ■  nX  x^z-NR2 

=  n(X  JL^-kx2). 

(4)  T  «  E  +  TR  , 

2  T 

(5)  s  ■  This  is  the  desired  solution. 

2  C 

(6)  The  pooled  estimate  Sp  *  . 

(7)  If  it  is  desired  to  test  the  null  hypothesis  for  equality  of  means; 

TR  , 
tr  s-jj-y  ,  and 


F 


with  (k-1)  and  (N-k)  degrees  of  freedom. 
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D.  Formulas  for  the  Variance  of  a  Total  Sample. 

It  is  simple  to  obtain  the  desired  formulas  for  the  variance  (and 
standard  deviation)  for  the  total  sample  by  following  the  procedures  of  the 
analysis  of  variance  given  in  the  previous  section.  These  formulas  are 
given  below; 


(1)  The  general  formula: 


2  E(ni-l)si2  +  En^x^-N*2 
B  = 


(IV) 


(2)  If  all  n^  are  equal; 

2  (n-l)E  Sj2  +  n(2ft^2-kfi2) 
■  "  “  N-l 


(V) 


(3)  It  a  pooled  estimate  of  the  variance  is  available  and  the  n 


are  unequal,  formula  IV  may  be  written  thus; 

,  (N-k)e  2  +  En/ft.2-NS2 
2  '  '  p  i  i 

*  N-l 


i 


(VI) 


(4)  If  a  pooled  estimate  of  the  variance  is  available  and  the  n^ 
are  all  equal,  formula  V  may  be  written  thus; 


(N-k)s  2  +  ntLft^-kS2) 


2  '  ’  p 

•  s  - 1. 


N-l 


(VII) 


(5)  If  k  ■  2  and  n^  a  n2,  formula  V  may  be  further  simplified: 


2  N-2 


(s 


2  ,  .  2\  N  .  v2 

1  +  *2  ^  +  4(N -1)  ‘  *l"X2  ‘ 


(VIII) 
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(  Tf  all  r>  a  /.mi 
'  '  - i  “*  ~  '-'V 

may  be  used  for  formula  V: 


•  —  J  —  J  _  1  _ 


'»«■  «■««>  iouowmg  approximation 


2  ~  2 
-  *A 


4Zi  ^  Ex^ 

Sj  “  ‘  *2  a  8p2  +  “TT  ■  #2* 


(IX) 


follows: 


In  Appendix  II,  it  is  shown  that  the  error  in  formula  IX  is  as 


Error  =  s 


l2-S2  =  jL.  (E  s,2-e2) 


(X) 


The  error  described  in  formula  X  is  always  positive. 

(7)  Formula  XI  is  offered  as  a  substitute  for  formula  IV  and 
formula  XII  as  a  substitute  for  formula  V.  Actually,  formulas  XI  and  XII 
may  require  more  labor  than  the  original  formulas,  but  they  will  usually 
involve  smaller  numbers  and  may  frequently  result  in  greater  accuracy. 


S(nl-l).i2  +En1(«i.ft)2 

N-l 

(XI) 

(a-lllsj2  +nr(xrS)Z 

N-l 

(XII) 

E.  Formulas  Associated  with  Changes  in  Data. 

(1)  Frequently,  after  computing  the  desired  statistics  for  a  sample 
of  size  n1(  the  worker  is  faced  with  the  necessity  of  adding  an  extra  group 

of  n2  observations  to  the  sample,  If  n^  is  large  and  n 2  relatively  small, 

it  would  appear  to  be  desirable  to  compute  the  mean  and  variance  for  the 
n2  additi°liai  observations  and  determine  the  statistics  for  the  entire 

sample  from  formulas  I  and  IV.  This  technique  is  illustrated  in  Appendix 
i.  n  *  * 


mmamsmsm  "*=  **■***&«  w«s»*w)**a.-  Hfs'r^  -■ 
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(2)  In  the  event  only  one  new  observation  (y)  has  been  added  to  the 
sample,  formulas  XIII  and  XIV  uflci  a  simple  procedure  for  obtaining  the 
desired  mean  and  variance.  Similarly,  formulas  XV  and  XVI  may  be  used 
if  two  observations,  (y)  and  (w)  are  to  be  added. 


n1‘  xx  +  y 

n^  +  1 


(XIII) 


(*!  -  y)1 

-r-rr 


(XIV) 


*y*i +  y +  w 


2  <n1-l)s12  +  y2  +  w2  +  jyilj2  *  (i^  +  2)S2 

"  “  nJ”+T 


(XVI) 


(3)  Similarly,  after  computing  the  mean  and  variance  for  a  sample  of 
sire  n^ ,  it  may  be  necessary  to  discard  n^  observations.  If  the  mean  and 

variance  are  computed  for  the  observations  which  have  been  discarded, 

formulas  XVII  and  XVIII  may  be  used  to  obtain  the  mean  and  variance  for 
the  remaining  observations. 


(XVII) 


»l  -  n2 

2  (nr1),i2  •  (»2-1>*2i  -  ivV**  •  Vj  +  Vi2  _ 

- - -  »2  ■  1) -  '  IXVIII) 

This  is  illustrated  in  section  E  of  Appendix  1, 

(4)  Discarding  One  Term 

If  there  is  only  one  term  (y)  to  be  discarded,  formulas  XIX  and  XX 
may  be  used. 
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nlXl  *  y 


(XIX) 


2 

8  3  - - 

V2 


(5)  Replacing  Observations 

If  a  group  of  n^  observations  in  a  sample  of  sis*  should 

by  changed,  one  may  follow  the  steps  discussed  in  sections  (1)  and  (3), 

If  it  is  only  one  observation,  formulas  XXI,  XXII  and  XXIII  may  be  used, 
Assume  y  is  the  value  to  be  removed  and  replaced  by  w, 


nixi  ■  y +  w 


(XXI) 


2  2  ,,2  _  2, 

2  2  w  -  y  -  v*  *  *1 ) 

B  =  S,  +  ,  — 1  ■■ 

1  n^l 


(XXII) 


s2  ■ « 2  +  (w-y)’  Kni-1)w  +  (ni +  !)y  ■ 

1  - (iqrtvi) - 


(XXIII) 


F.  Variance  and  Mean  for  a  Total  Population  Composed  of  k  Normal 
Populations 

It  appears  appropriate  to  conclude  with  a  brief  discussion  of 
population  parameters.  Assume  a  total  population  is  composed  of  k  normal 
sub  populations,  with  mean  p^  and  variance  7^;  and  each  contributing  to  the 

total  population  in  the  proportion  f. ,  with  Zf.  a  1.  Formula  XXIV  givas 

'  *  2 
the  mean  (p)  for  the  total  population  and  formula  XXV  for  the  variance  (7  ) 

of  the  total  population. 


p  *  f.p.  +  f.p,  +  .  .  .  +  f.  p. 


(XXIV) 


*aw  f  vyasftM  .ssspwwt 
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k  2 

S  iit  IT 

1  i  i 


+  -  w  , 


\aav; 


The  derivation  of  these  formulas  is  given  in  Appendix  III. 
The  chief  reason  for  including  this  section  ie  to  point  out  the  similarity 
between  formulas  XXIV  and  1  and  between  XXV  and  XI,  which  is  just  as 
would  be  expected. 

APPENDIX  I  -  Examples 
A.  Example  One  -  (All  n^  equal) 

(l)  Consider  the  example  given  by  Table  2  in  which  there  are 
four  equal  subsamples,  each  of  size  ten. 

TABLE  2 


SS(1) 

SS{2) 

SS(3) 

88(4) 

350 

300 

300 

300 

340 

295 

310 

275 

335 

310 

340 

280 

345 

315 

330 

310 

300 

305 

290 

305 

325 

325 

285 

290 

330 

285 

295 

260 

335 

310 

300 

325 

325 

32  5 

305 

290 

335 

330 

290 

280 

10 

10 

10 

10 

334, 00 

310.  00 

340. 50 

291.  5( 

243, 33 

205.  56 

319. 17 

361.  3' 

N  »  40 
8  =  310 
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2 

(2)  Using  ihe  raw  data  in  this  example,  the  value  s  ■  503.  85 
may  be  rnmrmted.  However,  it  is  the  purpose  of  this  example  to 

demonstrate  techniques  for  obtaining  *2  if  the  raw  data  is  unavailable  or 
if  N  is  so  large  that  it  would  not  be  feasible  to  use  the  raw  data.  The  first 
step  will  be  to  use  formula  II  to  obtain  S, 


=  310. 


(3)  Table  3  demonstrates  the  application  of  the  analysis  of  vari- 
ance,  as  described  by  Table  1,  to  obtain  s2, 


Sources  of  Variation 


Treatments 


TOTAL 


TABLE  3 


Mean  Square 


(k-1)  =  3  TR"  9,485  tr  »  3161.  67 


(N-k)  ■  36  £"10,165  Bp2"  282.36 


(N-l)  ■  39  T-19,650  s2  *  503.85 


F 


11.20 


Whe 

ire: 

E  « 

(n-l)  (Ssj 

2)  ■  10,165 

TR 

«  (n)  (SXj 

2)  .  N82  ■  9,485 

T  • 

E  +  TR  = 

19, 650 

.2  ■ 

T 

'  nnj  - 

503. 85 

e  " 

V'503.  85 

■  22.45, 

If  a 

pooled  estimate  of  variance  is  desired: 

•p2 

8  wzj 

>  282,  36. 
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If  f"r»  t*At  frtj*  tVi  •  enn>Ufy  n  f 


TR 

tr  =  -  3,161,  67 

tr 


F  o  — -  -  11.  20  with  3  and  36  degree*  of  freedom.  The  value 

■ 

p 

of  F  indicates  that  the  difference  in  means  is  highly  significant. 

(4)  Applying  the  formulas  from  Section  IH-D,  one  obtains: 


(a)  s 


2  (n-l)Z8l2  +  nJZ^2  .  k£2) 

"*  5m 


(v) 


(9)  (1129.45)  4  (10)  (385,348.50  .  4  x  96,1001  _  §  g5 

39 

(b)  If  a  pooled  estimate  of  the  variance  is  available,  one  may 
use  formula  VII. 

2  (N-k)s  2  +  n(Zft  2  -  k*2) 

s  >  - — i -  *  (VII) 

(36)  (282.  36)  4  (10)  (385,  348.  50  ■  4  x  96,100)  ^  ^ 


(c)  If  it  is  desired  that  the  number*  be  kept  smaller, 
XII  may  be  used. 

2  (n-l)!*^  +  nl(j|  -  5)2 

- - 5m -  " 


formula 


(XII) 


(9)  (1129.45)  +  (10)  (946.  50)  .  M 
39 
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k 


i 


! 

i 

j 


< 


! 

I 

i 
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(d)  If  an  approximation  is  desired,  one  may  use  formula  IX. 


s 


A 


2 


+ 

IT 


=2 

x 


1129.4?  +  385,  348.  50 
4 


96,100  =  519.49  ; 


giving  a  positive  error  of  15,  64,  exactly  what  formula  X  would  indicate  the 
error  to  be. 

B.  Example  Two  -  (n^  unequal) 

(1)  Consider  the  following  example  in  which  there  are  four 
•ubsamples  and  a  total  sample  size  of  32;  Table  4, 

TABLE  4 


SS(1) 

SS(2) 

SS(  3) 

SS{4) 

350 

300 

300 

300 

340 

295 

310 

275 

335 

310 

340 

280 

345 

315 

330 

310 

355 

305 

290 

305 

300 

325 

285 

290 

325 

285 

300 

330 

310 

325 

325 

330 

ni 

9 

10 

7 

6 

*1 

333, 89 

310.  00 

307. 86 

293.  33 

273.  61 

205.  56 

415, 43 

196,  67 

N  «  32 

x  *  313,125 

[ 


I 


.1 
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(d)  If  an  approximation  is  or.e 


+ 

IT 


*2 

x 


1129.41;  +  385,  348.  50 
4 


96,100  =  519,49  ; 


giving  a  positive  error  of  15,  64,  exactly  what  formula  X  would  indicate  the 
error  to  be. 


B.  Example  Two  -  j(n^  unequal) 

(1)  Consider  the  following  example  in  which  there  are  four 
subsamples  and  a  total  sample  sise  of  32;  Table  4, 

TABLE  4 


SS(1) 

SS(2) 

SS{  3) 

SS<4) 

350 

300 

300 

300 

340 

295 

310 

275 

335 

310 

280 

345 

315 

330 

310 

355 

305 

290 

305 

300 

325 

285 

290 

325 

285 

300 

330 

310 

325 

325 

330 

ni 

9 

10 

7 

6 

*i 

333,  89 

310,  00 

307. 86 

293.  33 

mm 

273,  61 

205,  56 

415.  48 

196,  67 

N  a  32 

x  =  313. 125 

J 


* 


«SWW®W«E*JS8(S!^**B» flWWBSWHeHS'"  1  j» 11 !  MWPW|l*?,liMl,l,?l 


Design  of  Experiments 


(2)  From  the  <">f  32.  the  value  s  =  452,  82  can  easily  be 


computed. 


(3)  Table  5  gives  the  analysis  of  variance, 

TABLE  5 


Sum  of 
Squares 


T  reatments 

(k-1)  =  3 

Error 

(N-k)  ■  28 

TOTAL 

(N-l)  =  31 

•p2*  268. 40 


T=14 , 044.  83 


Where: 


E  ■  1(^-1)  s12  o  7515.15 

TR  *  IiijUj2  -  Nx2  *  3,144,042.18  -  3,137,512.50  *  6529.68 
T  ■  E  +  TR  -  14,044.83 


a  453.  06. 


(Note  that  there  is  a  slight  difference  between  this  estimate  and  the  one 
obtained  from  the  basic  data,  due  to  rounding  errors,  ) 

s  »  V 453.  06  =  21.  29. 

If  a  pooled  estimate  of  variance  is  desired; 

•p2  -  w%  ■  268'4°- 


If  it  is  desired  to  test  the  equality  of  the  means  for  the  four 


no 


I^Coi gi't  \j k  iJA^c rime IK 8 


tr  *  ■  2176.  56 


*  8.11  with  3  and  28  degrees  of  freedom.  Thia 


indicates  that  the  difference  in  means  is  highly  significant. 

9 

(4)  Applying  the  formulas  from  Section  III-D,  one  obtains; 
E(n.-l)s  2  +  2n,x.2  -  Nx2 

(*)  *2  *  -  e'  -N-  -■  - -  ■  (IW) 

7515.15  +  3,144,042.18  -  3,137,512.50 
- — -  -  453.06. 

(b)  If  a  pooled  estimate  of  the  variance  is  available,  formula 
VI  may  be  used. 

,  {N-k)s  2  +  In.*.2  -  NS2 

s2  -  - 2 — nT^ -  •  <VI> 


(28)  (268.40)  *  3.144,042.18  •  3,137.512.  50 


453.11. 


(c)  If  it  is  desired  that  the  numbers  be  kept  small,  formula 
XI  may  be  used. 

£(n  -l).  2  +  r»  (i  -  i)2 

s  -  - tn -  -  (XI) 


7515.15  +  6523.42 


»  452.86. 


(Note  that  this  is  much  closer  to  the  true  value  than  those  listed  under 
a  or  b). 


C.  Example  Three  -  (k  =  2,n^  «  n^,) 

(1)  To  illustrate  formula  VIII,  the  first  two  columns  from 
Table  2  will  be  used.  From  this; 


:■*  tti-SXZhZS  .  -5  SEfifcW  W  *?■.. 
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2  _  (39)  (503.85)  +  (4)  ( 374.  80)  +  (40)  (.  51)2  +  (4)  (4.  09)2 
8  44 

■  482.42 

by  formula  XI. 

2 

(4)  Actually,  the  value  for  s  using  raw  data  ia  482.80. 

E.  Example  Five;  (Remove  observation*  from  a  sample  of  alee  n^.  ) 

(1)  The  data  of  example  four  will  be  used  for  this, 

=  45 

Xj  *  309. 49 
a^  =  482.  80. 

(2)  Remove  the  5  observations  which  were  added  in  example  four. 
n2  =  5 

x2  *  305. 40 

*22  *  374. 80. 

(3)  Use  formulas  XVII  and  XVIII,  giving; 

g  a  Ll5l(3.09.4^....(5a3_05,40}.  ,  qq 

2  (44)(482. 80)-(4)(374.  80)-(40)(  310)2.(5)(305,  40)2+(45)(  309.  49)2 

*  “  39 


504.  64. 
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APPENDIX  11  -  DETERMINATION  OF  THE  ERROR  IN  FORMULA  IX 

Using  formulas  V  and  IX,  the  following  error  is  observed: 

\ 


.  *  2  .2 
E  ‘  ’a  *  1  "  — “k -  '  * 


(n-l)I  ■  2  +  n(Ex  2  -  kS2) 


(N-nJEs^  nEx^  «2 

n(n-"i)  '  n(n-i)  +  nTT 

Es.2  (n-l)Es,2  nEx  2  ,  =2 

i  v  '  i  __i _  ,  kn»_ 

“  ~7T  '  N(N-l)  N(N-l"5  N(N-l) 
Error  *  sA2  -  s2  *--p  (Es^2  -  s2).  . 


a-— -w 


1 


i  2 

Inasmuch  as  Es^  is  larger  than  ■  ,  the  error  will  always  be  on  the 
positive  side. 


APPENDIX  III  -  DERIVATION  OF  THE  MEAN  AND  VARIANCE  FOR  A 
POPULATION  COMPOSED  '6fr  k  flbRMAL  POPULATIONS 


A.  Assume  each  of  the  k  normal  populations  have  a  mean  fx.,  variance 

r.  ,  and  contributes  to  the  total  population  in  the  proportion  f^,  with 

f,  +  f_  +  .  .  .  +  f.  *  1. 

12  k 


B.  y 


2cr 


1  ,  .2 


2ire. 


1 


1 _ .  ,2 

-tH1 

.  e  '  Zirk  I  , 

2-H7. 

k 


vi '  T-iti- 


C.  m  (6)  *  f^e 


1  2  2 

2  1  6  °-l  +*1 


1  2  2 
2  '  6  ffk  +  ^ 


+  f,  e 


Design  of  Experiment* 


116 


Design  of  Experiments 


(2)  Pooled  estimate  of  the  variance 

*p  ‘  N-k 

(3)  Variance  for  the  total  sample 

2  E(ni-l)si2  +  In^2  -  Nx2 

•  a  5TT 

2  (N-k)s  2  +  En.x  2  -  Nx2 
s  c  p _  1  i 

N-l 

2  1(^.1)^  +  In^  -  x)2 

- - FHI - 


C.  Formulas  associated  with  changes  in  data 

(1)  Add  an  observation  y  to  a  sample  of  site  n^ 

nl*l  +  y 


fi  * 


t\^  +  1 


2  ni  ‘  1 

s  ■ 


n, 


1 


2  +<v_lL 

*1  n,  +1 


1 


(2)  Add  observations  y  and  w  to  a  sample  of  sice  n, 

n  fij  +  y  +  w 

8  B  - n  7  Z  - 

2  (n^-l)s^2  +  y2  +  w2  +  n^2  -  (n^  +  2)x2 

s  . 


1 


Formula  Ill 


Formula  IV 


Formula  VI 


Formula  XI 


Formula  XIII 


Formula  XIV 


Formula  XV 


Formula  XVI 
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(3)  Discard  observations  from  a  sample  of  else  n^ 


jj 


X  = 


Vl  '  n2X2 

"I  ‘n2 


Formula  XVII 


2  (vi^2  "  ^n2"^B22  +  n^*!2-*2)  ■  n2(*2 2-sii) 

(n^  -  n2  -  1)  Formula  XVIII 

(4)  Discard  the  observation  (y)  from  a  sample  of  size 

“A  -f 


X  = 


-  1 


Formula  XIX 


2  “r1  2  V*ry> 

’  *  '  *1  ’  (Hj-lKnj-2) 

(5)  Replace  the  observation  y  by  w 
nl*l  "  V  +  w 


5  = 


n, 


1 


Formula  XX 


or 


22  /e2  ,  2, 

2  2  w  -y  -*!  ) 

■  ■  s .  +  - - - 

1  nl  ’  ^ 

2  2  (w-y)  [(n^ljw  +  (ni+l)y  -  2^3^] 

8  ■  B1  +  - -  * 


Formula  XXI 


Formula  XXII 


Formula  XXIII 


D.  Formulas  associated  with  a  total  population  composed  of  k  normal 
populations 

/l  f 

i _  ,  >2’ 

2  1 


Let  y  =  Tsr  '  • 


.  1  .  (x-n  )2) 

^  1  /+...+ 


f. 


1 


VTttT” 

k 


1  28*. 

.  e  \  k 
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"  T  i2  T  •  •  •  *  =  i,  then: 


14  =  V*1  +  f2“2  +  '  ’  '  *  fkt*k 


*  --  St 


Formula  XXIV 


Formula  XXV 


SYSTEM  CONFIGURATION  PROBLEMS 
AND  ERROR  SEPARATION  PROBLEMS* 

F red  S,  Hanson 

Plans  and  Operations  Directorate 
White  Sands  Missile  Range,  New  Mexico 


ABSTRACT.  Practical  geometric  criteria  and  optimization  methods 
are  needed  for  laying  out,  or  selecting,  multi-instrument  configurations 
for  flight  measurement.  The  problem  is  to  discover  -  and  demonstrate  - 
some  principles  that  are  at  least  in  the  right  direction.  A  general  solution 
should  be  possible  for  the  variation  of  uncertainty  of  intersection  location 
as  a  function  of  angle s -of -inter se ction  of  line s -of- sight.  It  might  also  be 
possible  to  calculate  the  optimum  ground-pattern  for  a  given  station 
density  and  missile  trajectory.  The  second  problem  is  to  develop  -  in 
detail  -  analytical  tools  for  separating  poeition-measurement  error, 
time-measurement  error,  and  lack-of-fit  of  a  given  polynomial  --  as 
these  errors  exist  in  undesigned,  but  redundant,  data.  Questions  con¬ 
cern:  the  validity  of  linearization  of  data  for  "his  purpose;  procedures 
for  calculating  lack-of-fit  of  polynomials  of  degrees  greater  than  one; 
limitations  In  conversion  of  regressions  to  analyses  of  variancs. 

INTRODUCTION,  This  paper  is  clinical  --  especially  in  the  sense 
that  it  is  not  completed  work. 

BACKGROUND.  Figure  1  is  a  White  Sands  Missile  Range  briefing 
chart,  It  shows:  the  principal  Range  (heavy  line);  the  part-time  exten¬ 
sion  (at  the  top);  and  the  White  Sands  Monument  (small  internal  area). 
Headquarters  •  and  the  main  launch  areas  -  are  at  the  lower  end  of  the 
Range. 

The  distinction  between  optical  and  electronic  tracking  instruments 
has  been  lost  in  this  black-and-white  print.  Optical  instruments  include; 
cinetheodolite s ,  telescopes,  fixed  cameras ,  and  ballistic  cameras.  Not 
every  station  is  shown.  For  instance,  there  are  several  hundred  pre¬ 
pared  sites  where  fixed  cameras  ran  be  set  up.  Electronic  tracking 
instruments  include:  radars,  dopplers,  and  mies-dietance  systems, 

Again,  not  every  station  is  shown.  (There  are  several  hundred  prepared 
sites  where  DOVAP  receivers  can  be  set  up.  )  The  gray  -  and  part-gray  - 
dots  are  telemetry  receivers, 


*Commenta  on  this  paper  by  some  of  the  panelists  can  be  found  following 
the  figures  at  the  end  of  this  article. 
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It  may  H»  apparent  that  the  systems  in  n8uxc  1  were  not  laid  out  on 
any  rigorous  basis, 

CONFIGURATION  HYPOTHESES,  More  than  three  years  ago  (Ref.  1), 
the  writer  asserted  two  hypotheses  about  instrument  layout,  or  selection  -- 
to  initiate  action  toward  solution. 

First,  it  was  asserted  -  intuitively  -  that  the  most  favorable  elevation 
angle  for  observing  a  missile  is  45°.  Second,  the  writer  stated  in  optimum 
ground-configuralion  -  for  each  integral  number  of  stations  -  with  respect 
to  a  single  point  in  space.  This  was  done  on  the  assumption  that  the  best 
intersection  of  line s -of- sight  from  two  stations  is  -  when  considered  by 
itself  -  90°.  Conversely,  it  was  assumed  that  the  worst  intersection 
occures  when  one  station  looks  over  another's  shouldor,  or  they  look  down 
each  others  throats  --  0°  or  180°,  parallel,  Referring  to  Figure  2,  the 
most  favorable  ground-configuration  for  optical  stations  was  asserted  - 
without  proof  -  to  be;  two-station  -  right-isosceles  triangle  with  missile 
at  apex;  three-station  -  equilateral  triangle  with  missile  at  center;  (in 
all  subsequent  cases,  missile  at  center)  four-station  -  any  four  corners 
of  equilateral  pentagon;  five-station  -  said  pentagon;  six-station  •  any 
Mix  corners  of  equilateral  heptagon;  seven-station  -  that  heptagon;  etc. 

The  (corresponding)  intersection  angles  are;  90°,  120°,  72°,  and  51.4°. 

For  twelve  or  thirteen  stations  •  a  tridecagon  -  the  angle  would  be  down 
to  27.  7°, 

DEMONSTRATION  OF  HYPOTHESES.  After  proposing  this  paper,  the 
writer  made  a  crude  approach  to  demonstrating  (the  validity  of)  these  sim¬ 
ple  hypotheses. 

Figure  3a  shows  the  asserted  two-station  optimum.  This  can  be  any 
plane  through  both  stations  and  the  missile,  The  diagram  represents  the 
90°  intersection  -  together  with  some  dispersion  index,  such  as  the  stand¬ 
ard  deviation. 

Figure  3b  is  an  enlargement  of  the  area  of  uncertainty.  We  are 
assuming  the  two  instruments  are  equally  precise.  Let's  approximate 
the  actual  error-ellipse  by  the  almost-square  in  Figure  3a  -  and  approxi¬ 
mate  that  by  the  square  in  Figure  3b.  The  horizontal  diagonal  is  a  mea¬ 
sure  of  the  combined  error-variance.  If  we  increase  the  intersection  angle, 
by  moving  the  stations  farther  apart  -  ox  by  lowering  the  missile  -  the 
horizontal  diagonal  will  lengthen.  Of  course,  the  vertical  diagonal  will 
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shorten,  tu» responding!*/.  In  it's  not  sound  practice  to  improve 

data  in  one  coordinate  by  making  it  worse  in  another.  (If  we  decrease  the 
intersection  angle  -  below  90°  the  horizontal  diagonal  .ets  smaller,  at 
the  expense  of  the  vertical  diagonal,  )  So ,  we  may  conclude  90°  is  the 
practical  optimum. 

Now,  we  have  shown  that  90°  is  the  optimum  intersection  In  any  plane 
thru  both  stations  and  the  missile.  The  plan  for  which  the  degradations 
from  this  optimum  will  be  the  same  in  its  horizontal  and  vertical  projec¬ 
tions  is  the  45°  plane.  On  the  basis  that  there  is  no  preferred  coordinate, 
we  have  demonstrated  the  hypothesis  regarding  the  optimum  elevation 
angle, 

If  we  choose  to  take  our  geometry  in  algebraic  form,  we  can  use  the 
law  of  cosines  to  calculate  the  horizontal  diagonal  (Figure  3b); 

2  2  2 

a  «  b  +  c  -  2bc  cos  9 

where  b  and  c  are  measures  of  the  two  observational  variances.  6  is 
approximately  90°.  To  see  the  effect  of  changing  the  intersection  from 
90°,  let's  replace  6  by  90°  +  a  : 

a2  a  b2  +  c2  -  2bc  cos(90°  +  a) 

In  our  case,  b  and  c  are  equal,  so: 

a2  =  2b2  -  2b2  cos(90°4a) 

=  2b2  [1  -  cos(90°  +  a)] 

Substituting,  a2  =  2b2  (1  T  sin  a), 

So,  approximately,  if  the  intersection  angle  is  changed,  the  combined 
variance  in  one  coordinate  increases  as  the  sine  of  the  angular  deviation 
from  90°. 

A  similar  exercise  can  be  gone  thru  for  the  3-station  equilateral 
triangle.  In  that  case,  the  error-ellipse  is  approximated  by  an  almost- 
equilateral  hexagon. 
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_ _  . .  .  .  a  t  sat  rtTTnM  W  F.  Mammack  (Ref.  2)  has  fur* 

nished  the  writer  a  solution  which  itoes  not  depend  on  approximating  the 
almost -square  --  or  on  testing  a  hypothesis. 

Referring  to  Figure  4a  -  the  trigonometry  for  the  general  two -station 
case  yields: 


b  ain(9i  -  6 z) 
X  ~  1  sin(  0  ,  +  6  p 


.  sin  9i  sin  6; 
^  ”  aini0j  +  &2J 


Applying  the  standard  error -propagation  formula: 


8x  \  2  |9x  \  2 

801  |  ,01  \  ® ®  2  )  6  2 


(and  similarly  for  y)  yields; 


2 

<  = 

x 


iin4(01  +  6  2) 


H  »in*Ue2.^  +  4- 


,2  /  4  2  4  2 

- * -  <  v  sin’9  _«  Q  +  »in  0.«e 

4,  \  2  0.  * 

iin(01+02)  '  1  1 


Simplifying  to  the  equidi.tant,  equal-precision  case  (Figure  4b): 


2  2.2.2 


S  +  .  »  b  (1  -  cos  20! 

X  V  J - A 

sin  26 


If  this  total  error  is  minimised  with  respecv  to  0  ,  the  minimum  is  found 


to  occur  at: 


20  =  1/- 


r  70.  5' 


26 
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So,  Mimmack's  optimum  intersection  angle  is  109-  5°. 


12  3 


In  R,  C.  Davis'  wots  report  on  his  cinetheodolite- reduction  method 
(Ref.  3),  he  minimized  the  observational  error-ellipse  of  the  two-station* 
missile  triangle,  by  a  matrix  process.  With  the  stations  fixed  and  the 
missile  altitude  allowed  to  vary,  Davis  found  the  optimum  intersection  to 
be  120°.  He  theorized  this  was  the  result  of  compromise  between  the 
most  favorable  intersection  and  the  decrease  in  the  linear  error  (corre¬ 
sponding  to  a  given  angular  error)  as  the  missile  moves  closer  to  the 
stations,  Mimm&ck's  solution  represents  this  same  c&se.  So,  there  is 
an  apparent  discrepancy  in  their  results. 

With  the  missile  altitude  fixed  and  the  stations  free  to  move,  Davis 
found  the  optimum  intersection  to  be  60°.  He  theorized  this  was  the 
result  of  compromise  between  moat  favorable  intersection  and  moving 
the  stations  closer  to  the  missile.  The  present  writer  thinks  Davis' 
explanations  are  correct. 

However,  it  appears  that  the  optimum  ground-configurations  hypoth¬ 
esized  in  this  paper  are  still  optimum  when  the  effect  of  slant  range  is 
included,  Also,  45°  planes  are  the  only  ones  for  which  the  degradations 
(of  coordinate  projections)  from  the  optimum  intersection  will  be  the 
same  -  whatever  the  optimum  may  be.  So,  we  have  "demonstrated"  a 
simple  set  of  rules  for  laying  out,  or  selecting,  a  group  of  stations  - 
for  any  given  point  on  a  missile  trajectory  and  for  determining  the 
optimum  scale  of  their  configuration.  The  point  used  could  be  the  mid¬ 
point  of  a  trajectory  segment. 

MINIMUM  BIAS  CONFIGURATION.  The  demonstration  based  on  Fig¬ 
ure  3  treated  error  as  a  dispersion  index  (or  precision  index).  Let's 
consider  (it  as)  a  discrete,  or  net,  error.  Then,  in  Figure  3a,  if  we 
increase  6  above  90°,  the  horizontal  (error-)  resultant  -  corresponding 
in  size  to  the  smaller  almost- square  •  will  lengthen  if  the  (discrete 
angular)  errors  happen  to  have  the  same  sign  (Figure  3a);  if  the  errors 
have  °£££lilg  signs  (Figure  5b),  their  (vertical)  resultant  will  shorten 
correspondingly.  (Of  course  -  in  the  equal-accuracy  case  -  there  will 
be  only  a  horizontal,  or  only  a  vertical,  resultant. )  In  general,  it's  not 
sound  practice  to  (eet  out  to)  improve  data  in  one  coordinate  by  taking 
an  even  chance  that  we  will,  instead,  make  it  worse  in  another,  (Even 
chance,  because  -  to  the  extent  that  a  given-type  instrument  consistently 
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has  the  same  bign,  it  is  more  likely  to  be  adjusted,  or  corrected  for,  ) 

If  we  decrease  6  (below  90°) ,  the  possible  homopolar  (horizontal)  error- 
rcsultant  gets  ■mailer,  ttL  i'ne  expense  of  me  possible  heteropolar 
(vertical)  error -resultant  gets  smaller,  at  the  expense  of  the  possible 
heteropolar  (vertical)  error -resultant,  So,  we  may  conclude  -  90°  is 
the  practical  optimum.  The  rest  of  the  writer's  geometric  and  algebraic 
demonstrations  apply  similarly,  Summary:  perpendicular  intersection 
(per  se),  45°  elevation,  the  right-isosceles  triangle  for  the  two-station 
case,  etc,  are  all  optimum  for  accuracy  as  well  as  precision. 

PATTERN  HYPOTHESES.  How  does  one  generalise  from  a  tingle 
group  of  stations  to  a  larger  area  --  for  (several  segments  of)  a  family 
of  trajectories  ?  What  sort  of  patterns  can  we  construct  with  our  optimum 
figures  ?  In  Figure  6,  what  is  wrong  with  a  grid  built  up  of  optimum  three- 
station  configurations  ?  Equilateral  trlangelt  form  hsxagons,  which 
violates  our  odd-sided  rule.  Each  station  is  in  line  with  all  the  other 
stations.  Continuing  in  Figure  6,  pentagons  seem  to  form  a  desirable 
pattern  -  leaving  a  few  gaps  of  isosceles -triangle  pairs,  (Four  stations 
are  In  line  across  each  triangle  pair.  )  Heptagons  might  do  as  well, 

In  determining  the  optimum  layout,  the  decisive  constraint  could  be 
the  number  of  stations  needed  to  meet  requirements  (for  precision),  Or, 
it  could  be  budgetary  (the  number  of  stations  permitted  per  hundred  sq, 
mi.).  Or,  it  could  be  the  effective  range  of  a  station  -  as  a  configuration 
radius. 

Perhaps  someone  can  demonstrate  that  the  optimum  pattern  is  ran¬ 
dom.  Or,  that  a  random  pattern  is  not  optimum.  A  random  pattern  might 
have  the  minimum  percent  of  stations  in  line  with  each  other  -  but  it 
wouldn't  be  the  moat  efficient  dispersion.  Mimmaek  (Ref.  2)  notes  that 
It  is  desirable  for  a  position  measurement  to  be  independent  of  any 
coordinate  system;  that  this  implies  the  station  geometry  should  be  free 
of  symmetries;  that  the  symmetry  of  being  in  the  same  ground-plane  is 
largely  unavoidable. 

DISCUSSION  OF  CONFIGURATION.  The  optimum  configuration 
would  maximize:  accuracy,  precision,  versatility,  reliability,  and 
economy.  Flight-measuring  instruments  exist  in  three  conditions:  fixed, 
(self-contained)  mobile,  transportable  (to  prepared  sites). 
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The  writer  chose  to  start  with  the  precision  of  a  single  pomt-in- 
apace,  because  this  is  WSMR'i  operating  standard  -  and  because  it  lend# 
itself  to  an  analytical  approach  which  proceeds  from  the  simple  to  the 
complex,  The  Range's  instrumentation  plans  are  prepared  per  segment 
of  a  trajectory.  The  present  standard  seems  to  be  the  best  (single) 
compromise  between  an  operating  viewpoint  and  a  missile -engineer  view¬ 
point.  Aside  from  having  a  consistent  benchmark,  the  important  question 
is;  "What  aspect  of  a  given  missile -performance  variable  is  most  signif¬ 
icant  to  a  particular  missile  project?" 

This  is,  after  all,  a  clinical  paper,  The  writer's  aim  is  not  - 
necessarily  «  to  solve  the  whole  problem  by  an  analytical  approach,  (It  is 
to  increase  understanding  of  the  subject.)  Ws  "demonstrated"  the  "90°  - 
optimum"  intersection  in  any  plane  -  for  observing  a  point -in -space,  We 
found  a  (limited)  approximate  solution,  in  two  dimensions,  for  the  varia¬ 
tion  of  uncertaii.t /-of-intersectlon-location  as  a  function  of  angls-of-inter- 
eection-of-linea-of-sight.  Mimmack  (Ref,  2)  obtained  a  general  solution 
(to  this  problem)  for  two  dimensions;  his  method  could  be  extended  to 
three  dimensions,  It  may  be  that  an  optimum  ground -pattern  can  be  con¬ 
structed  with  pentagons. 

The  optimum-overall-pattern  problem  could  be  stated:  "Is  there  a 
unique  solution  for  the  most  efficient  layout,  for  a  given  optical -station 
density  -  or  for  a  given  effective  station-range  -  and  for  the  Range's 
total  trajectory-volume  ?  "  It  teems  clear  that  any  thorogolng  analysis 
of  this  problem  must  be  made  in  three  dimensions, 

Reference  4,  revised  annually,  discusses  computer  programs  for 
propagating  "typical"  errors-of-observation  thru  the  (trigonometric) 
equations  relating  coordinates  of  any  given  point-in- space  to  the  (angular, 
etc,)  "observations"  of  the  point  by  stations -of -known-location.  Thase 
are  essentially  the  same  programs  used  for  trial-and-error  simulation 
at  White  Sands,  AMR  (now  ETR)  calls  the  -  a  priori  -  error  estimates 
so  obtained  "a  geometric  dilution  uf  precision  (GDOP)".  Properly,  this 
term  should  be  reserved  for  the  geometric  component  of  position-mea¬ 
surement  variance. 

ERROR  SEPARATION  PROBLEM,  The  second  problem  is  this: 

"Can  we  determine  (by  statistics^,  methods)  -  qualitatively  and  quantita¬ 
tively  •  how  much  of  the  error-variance  in  our  (final)  missile-position 
data  is  position-error,  and  how  much  i»  time-error  ?  "  For  velocity  and 
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acceleration  (or  smoothed  position  data),  we  would  also  like  to  know  the 
relative  magnitude  ot  a  third  variance  component  -  the  lack-ol-lit  oJt  the 
polynomial  which  we  use  to  obtain  (smoothed  and)  derivative  data. 

The  jitter  (and  wander)  of  time-signal  generators  is  email.  Propa¬ 
gation-  and  receiver-delays  are  appreciable  •  different  for  each  station  • 
somewhat  variable  -  and  partly  compensated  for.  Recording  delays  for: 
time -code  marks,  missile  image,  (angular)  dial  readings,  etc.  are  appre¬ 
ciable,  different,  and  somewhat  variable,  Overall  time -measurement 
error  includes  errors  in  synchronizing:  timing,  miBsile  position,  and 
mount  position  --  physically,  on  the  record,  in  conversion,  in  computing, 
and  in  reporting. 

For  a  Mach  10  missile,  a  millisecond  overall  time-measurement 
error  would  be  equivalent  to  a  position  error  of  10  ft.  A  recent  figure 
for  the  speed  of  an  ICBM  warhead  is  26,400  ft/sec  (Ref.  5);  in  that  case 
a  millisecond  is  26.4  ft. 

Actual  requirements  -  and  capabilities  -  for  instrumentation  timing- 
and- synchronization  should  be  known  -  in  specifiable  terms.  A  complete 
description  of  position  accuracy  -  or  precision  -  would  include  a  separate 
specification  of  time  accuracy  -  or  precision.  If  time-measurement  error 
is  ignored,  it  shows  up  as  position  error  •  but,  it  cannot  be  decreased  by 
improving  the  position-measuring  device  (as  such).  If  time-measurement 
error  is  appreciable,  these  two  components  of  position  error  should  be 
separated  before  calculating  velocity  (or  acceleration)  error,  We  don't 
know  that  time-measurement  error  is  an  appreciable  part  of  the  whole  - 
but  we  can't  afford  not  to  know  how  much  it  is. 

This  paper  presents  problems  --  not  solutions.  But  -  in  presenting 
this  problem  -  let's  review  the  approaches  the  writer  has  already  considered. 

SEMI -QUANTITATIVE  SEPARATION.  About  four  years  ago,  the  writer 
suggested  a  semi -quantitative  method  for  "separating"  time  error  from 
position  error  -  in  final  data,  Let's  look  at  the  three  types  of  "regression" 
(correlation)  of  a  position  coordinate  and  time  (Figure  7), 

Figure  7a  shows  regression  of  x  as  a  function  of  t  -  in  which  time  is 
assumed  to  be  exactly  measured,  and  that  curve  is  fitted  which  minimizes 
the  (sums  of  the  squares  of)  the  deviations  in  position,  This  is  the  one 
WSMR  uses,  in  its  data  reduction, 
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t  igure  7b  shown  rcyiciilw'.  cf  t  as  ■  fur'c*’*'-'"  of  x  -  in  which  position 
is  assumed  to  be  exactly  measured,  and  that  curve  is  fitted  which  minimises 
the  (sums  cf  squares  of)  the  deviations  in  time.  From  a  mathematical 
standpoint,  this  is  as  logical  as  the  first, 

Figure  7c  shows  simultaneous  regression  of  x  and  t  -  in  which  they  are 
assumed  to  be  measured  equally  well,  and  that  curve  is  chosen  which 
minimizes  the  (ss  of)  the  deviations.  This  is  sometimes  called  the  ''best 

fit". 


If  measurements  of  x  and  t  are  about  equally  in  error,  curve  c  will 
(tend  to)  fall  about  halfway  between  a  and  b  -  and  is_  the  best  choice,  in 
this  case, 

If  one  variable  is  badly  measured,  the  curve  which  minimises  ths 
variability  of  the  badly  measured  variable  will  (tend  to)  deviate  the  most 
from  the  other  two  --  but  will  (tend  to)  be  closest  to  the  (physically)  true 
relationship.  This  Justifies  use  of  method  a  (by  WSMR)  -  if  the  assump- 
tlon  that  position  is  (always)  much  more  poorly  measured  proves  correct. 
The  curve  of  . "best  fit"  >  c  •  best  represents  the  data,  as  such,  in  any 
case. 

By  comparing  these  three  types  of  regression  -  and  taking  into  account 
any  knowledge  of  the  (physically)  true  curve  from  Independent  data,  and/or 
physical  theory  --  it  is  possible  to  obtain  aemi-quantitatlve  estimates  of 
how  relatively  well  two  variables  are  measured.  The  writer  knowe  from 
experience  'hie  works  in  applying  linear  regresaion  to  rather  poor  data. 

It  may  be  an  even  sharper  tool  in  applying  curvilinear  regression  to  rather 
good  data. 

QUANTITATIVE  SEPARATION.  On  the  basis  of  redundancy  in  mea¬ 
suring  missile  position,  these  three  regressions  can  be  converted  to 
corresponding  analyses  of  variance.  Thie  ehould  permit  quantitative 
separation  of  time  error  and  position  error,  Procedurei  are  available 
for  analysis  of  variance  of  type*  a  and  b  regression.  Type  c  regression 
could  be  handled  -  for  the  linear  case  *  by  these  same  (single -fixed- 
variate)  methods,  by  a  rotation  of  axes.  It  may  also  be  possible  to  dis¬ 
cover  (or  devise)  a  bivariate  analysis  -  at  least  for  the  linear  case.  If 
necessary  curvilinear  data  can  be  transformed  to  linear. 
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Such  analyses  of  variance  include  a  lack-of-fit  term,  which  is  avail¬ 
able  for  the  linear  fixed- variate  case  in  Reference  6,  It  appears  to  be 
available  for  the  curvilinear  fixed- variate  case  from  (such  sources  as) 
References  7  and  8. 

The  usual  procedure  at  WSMR  is  to  fit  a  second-degree  polynomial, 

If  our  lack-of-fit  proves  to  be  appreciable  compared  to  position-error, 
it  will  follow  that  we  need  to  improve  our  data- reduction  procedure. 

The  writer's  questions  with  regard  to  the  above  analyses  of  variance 
are  these: 

1,  What  analysis -of- variance  components  can  we  get  from  linear 
fixed-variate  regressions  of  types  a  and  b  if  we  have  (apparent)  redundancy 
in  (a  given)  position  (coordinate)  at  (equally- spaced)  apparent  times  •• 

and  (if  we)  convert  these  asaumed-x  redundancies  to  assumed-t  redun¬ 
dancies  by  (means  of)  the  reciprocal-of-the-slope  of  the  type  a  regression 
(i.  e,  ,  if  we  multiply  by  the  corresponding  value  of  At/Ax),  Specifically, 
can  we  eeparate  timing-error,  position-error,  and  (two)  lack-of-fit  terms? 
As  a  working  reference  for  this  would  the  Panel  recommend  Reference  9  - 
or  some  other?  Same  questions  for  curvilinear  case  --  using  the  reciprocal 
of  the  type  a  slope  at  each  point  to  convert  •  and  substituting  Refsrsnco  10 
as  a  working  source. 

2,  Suppose  we  apply  this  fixed- variate  analysis  to  type  c  linear 
regression  by  a  rotation  of  axes  -•  and  calculate  the  as  turned -normal  re¬ 
dundancies  by  interpolating  between  the  usumed-x  and  the  (corresponding) 
asaumed-t  redundancies,  above  (in  proportion  to  the  ratio  of  the  angle- 
between-the-x-axis -and-normal  to  90°).  Can  we  get  anything  out  of  this 
transformed  type  c  analysis  of  variance? 

3,  Can  the  Panel  give  a  reference  which  shows  how  to  calculate 
lack-of-fit  for  type  c  linear  regression? 

4,  Can  the  Panel  give  a  reference  to  -  or  device  -  a  bivariate 
analysis  of  variance  for  linear  regression  if  we  have  (apparent)  redundancy 
in  (a  given)  position  (coordinate)  at  (equally- spaced)  apparent  times? 

Same  question  for  curvilinear  regrsssion. 
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5.  Suppose  we  transform  a  variable  to  linearise  a  (curvilinear) 
r*or*«»inn  and  then  perform  the  (linear)  analysis  of  variance  under 
question  1.  Is  it  necessary  to  leave  the  result  in  the  transformed  state? 

Is  it  valid  to  "untr&nsform"  the  variance  of  the  transformed  variable? 

Can  the  Panel  give  a  reference  on  estimating  the  error  due  to  "untransform¬ 
ing 11  ? 


6.  Does  Reference  7,  8,  or  10  clearly  give  a  procedure  for  cal- 
culating  lack-of-fit  for  curvilinear  single-fixed-variate  regression?  If 
not,  can  the  Panel  give  a  reference  which  does? 

SEPARATION  AT  A  POINT.  So  far  we've  taken  a  time -varying  look 
at  the  flight-measurement  process,  White  Sands  is  also  Interested  in 
(knowing)  the  uncertainties  associated  with  single  values  of  unsmoothed 
data,  It  should  be  posaible  to  make  a  hypothetical  -  if  inconclusive  • 
analysis  of  the  errors  of  a  single  point  (in  space  and  time)  by  looking  at 
the  error  as  all  (in)  position,  all  (in)  time,  all  tangential,  or  all  normal. 

An  additional  approach  to  the  "instantaneous"  aspect  might  be  to  consider 
(two)  successive  data-points  as  observations  of  their  mean  point.  Can  we 
get  any  -  qualitative  or  quantitative  -  separation  of  timing  and  position 
error  out  of  these  approaches?  Can  the  Panel  suggest  any  further  approach 
to  analysis  of  the  errors  of  single-v&lues-of-unlmoothed-data? 
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COMMENTS  ON  PRESENTATION  BY  FRED  HANSON 


F rank  E .  Grubb* 

Army  Ballistic  Research  Laboratories 
Aberdeen  Proving  Ground,  Maryland 

In  my  opinion  the  problems  and  questions  Dr,  Hanson  raised  can  be 
solved  satisfactorily  only  by  competent  personnel  working  rather  full  time 
on  the  overall  problem.1  I  say  this  because  the  problem  is  so  involved 
from  both  the  physical  and  the  analytical  standpoints  that  it  is  easy  to 
overlook  the  importance  of  all  of  the  'errors"  operating  simultaneously, 
so  to  epeak, 

Concerning  station  location  geometry,  I  think  that  something  can  in¬ 
deed  be  done  on  this  and  Dr.  Hanson's  ideas  may  be  near  enough  the 
optimum,  considering  other  involved  difficulties.  I  can  see  that  Whits 
Sands  might  decrease  position  estimation  errors,  etc,  ,  by  optimum  station 
locations,  whereas  the  Atlantic  Missile  Range  cannot  really  do  this, 

Just  what  sums  of  squares  must  be  minimised,  as  Dr.  Hanson  points 
out,  involves  considerable  study,  From  my  limited  experience,  1  have  the 
feeling  that  relative  time  is  quits  good  but  that  position  data  is  not  so  good 
because  of  intersection  geometry,  and  the  errors  which  creep  into  this  de¬ 
pending  on  unexplainable  biases  for  the  missile  flight,  calibration,  refrac¬ 
tion  and  other  corrections,  etc.  Of  course,  all  of  these  things  vary  with 
the  type  of  instrumentation,  etc. 

Power  spectral  density  type  analyses,  are  certainly  being  looked  Into 
by  many  people  now  and  thie  work  la  no  doubt  paying  off  as  many  of  the 
problem*  involved  neceesarlly  fall  in  this  area,  even  though  this  is  an 
added  dimension  of  complication, 

The  nearest  publication,  as  Dr.  Hannon  is  aware,  which  1  think  is 
beginning  to  approach  methods  required  to  settle  some  of  the  questions 
Dr,  Hanson  is  raising  is  the  annual  report,  "Accuracy  of  AMR  Instru¬ 
mentation",  by  H,  P,  Mann,  The  latest  version,  as  Dr,  Hanson  knows, 
does  contain  a  lot  of  good  material  and  attempts  to  cover  most  of  the 
important  viewpoints,  but  still  doesn't  go  far  enough. 

I  think  the  tracking  data  analysis  problem  is  by  far  the  most  interest¬ 
ing  overall  one  I  have  been  introduced  to  in  recent  years,  but  unfortunately 
it  is  something  that  does  not  carry  the  proper  priority  with  many  of  us  in 
•  pite  of  its  great  importance.  Our  Panel  on  Tracking  Data  Analysis  is 
quite  inactive  now  but  if  anything  comes  up  on  this  in  the  future,  I  wvuld 
hope  to  be  in  touch  with  Dr,  Hanson. 


COMMENTS  ON  PRESENTATION  BY  FRED  HANSON 


■EL' Til  11  XI.  U  C*  UQ 

Institute  of  Science  end  Technology 
The  University  of  Michigan 
Ann  Arbor,  Michigan 

Be fiy v*  commenting  on  Dr,  Hanson's  two  problems,  I  will  first  take 
up  the  m  v.iar  of  references,  I  certainly  recommend  F,  S.  Acton  and 
K,  A,  BrownU-e  (titles -Dr,  Hanson  mentioned),  Dr,.  Hanson  has  also  used 
Anderson  and  Bancroft,  which  is  good,  Further,  I  will  mention  E.  J. 
Williams'  "Regression  Analysis",  J,  Wiley  k  Sons,  and  Plackett's  "Regres¬ 
sion  Analysis",  Oxford  Press,  Also,  O,  Kempthorne's  "Design  and  Analysis 
of  Experiments"  and  H,  Scheffe's  "Analysis  of  Variance"  may  prove  useful. 
There  is  a  book  by  an  Australian,  P.  G  .  Guest,  "Numerical  Methods  of 
Curve  Fitting",  Cambridge  University  Press.  1961.  Perhaps  Dr.  Hanson  . 
should  look  at  the  symposium  publication,  "Tims  Series  Analysis",  SIAM 
Series  in  Applied  Mathematics,  J.  Wiley  it  Sens,  1963. 

Now,  to  Dr,  Hanson's  problems,  Number  1  first,  Certainly,  I  must 
comment  that  my  experience  with  the  NORC  project  at  Ft,  Monroe,  1941-42, 
and  with  the  Anti-aircraft  Artillery  Board,  Camp. Davis,  1942-44,- is  ancient 
history  by  comparison  with  the  stats  of  the  art  in  the  60s.  Generally,  I 
agree  with  Dr.  Hanson's  analysis  of  the  geometry  of  the  situation,  i.  e,  , 

45  degree  elevation  for  Une-of-iight  and  nearly  orthogonal  to  missile  path 
for  a  "reasonable"  interval  of  time,  From  the  algebra  associated  with 
the  geometry  one  should  be  able  to  work  out  the  error  propagation  for  the 
position  determinations,  Of  course,  one  must  keep  in  mind  the  "best" 
physical  model  for  the  flight  path  of  the  missile  in  using  the  observed  data 
to  obtain  be st  apparent  position  of  missile  at  a  given  time. 

I  tend  to  think  cf  this  first  problem  more  in  practical  considerations, 
given  that  the  technical  problem  of  determining  location  has  bean  resolved 
to  a  useful  accuracy  and  precision,  Some  method  of  assigning  priorities 
to  each  day's  or  each  week's  missions  must  be  worked  out,  Then  with  the 
resources  at  hand,  tn  allocation  must  be  made  of  stations  to  be  manned 
with  selected  equipments.  Consider  Figure  1  for  Mission  A  (highest  prior¬ 
ity),  Enough  paired  stations,  a  and  s',  b  and  b1,  etc.  ,  mutt  be  manned  to 
keep  this  missile  path  under  adequate  surveillance.  Now,  if  a,  b,  c  and  d, 
etc.  ,  are  too  far  apart,  there  will  be  too  much  uncertainty  in  the  computed 
positloni  in  the  halfway-between  regions.  Next,  Mission  B  (second  prior¬ 
ity)  has  to  be  similarly  supported  at  a  desired  minimum  level,  If  launch 


time#  can  be  programmed  to  some  extent,  it  may  be  that  some  manned 
stations  can  support  more  than  one  mission.  Continue  for  say  two  more 
Missions  C  and  D.  If  any  resources  are  left  over,  consider  increasing 
density  of  manned  pairs  for  Missions  A,  B,  C  and  D  in  that  order  to  shore 
up  obvious  weaknesses  in  trajectory  assessment.  These  practical  con¬ 
siderations  seem  much  more  relevant  to  me  than  going  into  geometrical 
considerations  beyond  the  triangle,  If  is  recognised  that  my  sketch  implies 
using  rectangles  or  quadrilaterals  in  assessing  position.  When  launch 
times  are  adequately  separated  so  that  all  manned  stations  for  each  of  the 
four  mie lions  can  track  each  launch,  thin  further  geometrical  coneidera-' 
tions  may  be  taken  into  account  along  the  lines  Dr.  Hanson  has  discussed. 

Now  1  turn  to  the  second  problem  of  analysis.  Yee,  one  would  like  to 
have  variance  components  for  timing  error  and  for  poeitiqn-measuring 
error.  But  how  can  one  separate  them?  Without  considerable  study,  more 
than  1  can  give  at  this  time,  1  have  no  direct  suggestion,  It  is  hoped  that 
Dr.  Hartley  has  given  Dr.  Hanson  some  useful  direct  suggestions.  I  use 
the  term  indirect  for  my  ideae  because  I  wieh  to  lean  on  "design  of  experi¬ 
ments"  considerations,  By  direct  suggestions  1  mean  extracting  from 
present  method  of  collecting  data,  components  of  variance  of  the  two  kinds 
desired. 

In  directing  Dr,  Hanson's  attention  to  design  of  experiments  concepts, 

I  believe  WSMR  is  in  an  outstanding  position  to  carry  out  some  special 
studies.  Of  course,  these  activities  must  be  budgeted,  but  it  does  not 
seem  unreasonable  to  program  some  percent  of  the  WSMR  annual  budget 
for  R&D  on  its  own  job.  What  the  percent  should  be,  I  don't  know,  but 
2%,  5%  or  7%  seems  reasonable.  Electronics  and  A/C  firms  do  better, 
What  kinds  of  experiments  one  asks  ?  On  some  missions  WSMR  may  have 
enough  spare  resources  so  that  it  can  double  up  on  position  measurements, 
i.  e,  ,  re  Figure  1,  again,  put  two  equipments  at  each  location  b,  b' ,  c,  c1, 
•ay,  I  assume  that  timing  errors  would  be  nearly  equal  at  any  single  loca¬ 
tion,  The  smoothed  apparent  poeition  data  (after  averaging)  should  then 
indicate  something  about  possible  "timing  component"  of  error,  If  a 
competent  person  in  design  of  experiments  were  to  spend  3-6  months  at 
WSMR,  it  seems  reasonable  that  other  experiments  with  useful  treatment 
combinations  could  be  suggested  and  suitably  designed  within  WSMR's 
resource  frame  work, 
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With  respect  to  the  orthogonal  regression  line,  there  !•  uuLuiug  in 
the  literature  that  I  am  aware  of  on  sampling  theory  for  the  regression 
coefficient  or  for  predicted  points.  A  general  reference  1  recommend  is 
J.  B.  Coleman,  Armais  of  Math.  Stat.  3,  79  (1932).  In  1963,  I  did  some 
work  on  the  design  of  a  flight  program  carried  out  in  Arizona,  By  flight 
replication,  we  were  able  to  obtain  sampling  error  information  about  the 
orthogonal  regression  coefficient  and,  thus,  overcome  the  lack  of  sampl¬ 
ing  theory  based  on  an  Internal  estimate  of  error, 

Further,  s  both  Prof,  Lieberman  and  I  have  pointed  out,  there  are 
no  difficulties  in  obtaining  an  analysis  of  variance  including  a  goodness- 
of-fit  term  even  though  the  regression  fitted  is  polynomial  or  otherwise 
non-linear,  so  long  as  the  least  squares  equations  are  linear  in  the 
unknown  parameters  to  be  estimated.  For  the  non-linear  least  squares 
equations  cases,  which  might  arise  from  a  physical  model  of  the  missile 
flight  path,  I  suggest  Prof.  Hartley's  recent  paper  in  Blometrlka,  51,  347 
(Dec.  1964). 

At  1ST,  we  have  a  quite  general  purpose  regression  program  which 
is  due  to  Dr.  Wyman  Richardson,  Also,  Robert  O.  Bennett,  Jr,  and 
myself  are  working  on  a  packaged  set  of  sub- routines  which  can  be  used 
for  doing  Analysis  of  Variance  type  calculations.  Perhaps,  Dr.  Hanson 
should  visit  us  to  get  information  on  these  programs,  Both  programs 
operate  on  IBM  7090  within  University  of  Michigan  Computer  Center 
Executive  System. 

No  doubt  WSMR  is  studying  the  application  and  use  of  the  newer  high 
accuracy  oscillators  for  its  timing  standards.  Could  not  these  "atomic 
clocks"  help  resolve  some  of  its  "timing  error"  problems  ?  Any  WSMR 
comment  on  the  use  of  these  oscillators  will  be  of  interest  to  us  at  1ST, 
since  we  are  studying  their  employment  for  networks  even  more  widely 
distributed  than  those  in  the  WSMR  systems, 


AN  EXPERIMENT  IN  MAKING  TECHNICAL  DECISIONS 
USING  OPERATIONS  RESEARCH  AND  STATISTICAL  METHODS 


Andrew  H,  Jenkins 
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and 

Edwin  M.  Bartee 

University  of  Alabama  in  Huntsville, 

College  of  Engineering ,  Huntsville,  Alabama 


ABSTRACT,  This  paper  presents  a  case  where  decisions  are  reached 
and  recommendations  were  made  on  a  multi-disciplined  technical  research 
program.  The  decisions  were  made  on  the  basis  of  a  technical  survey  using 
operations  research  techniques  and  statistical  methods  for  evaluation  rather 
than  a  rigorous  technical  evaluation  of  all  disciplines.  The  paper  presents 
the  technique  used  and  discusses  the  practical  limitations  of  the  method, 

I,  INTRODUCTION,  The  engineer  and  scientist  in  government  research 
programs  are  often  Required  to  make  decisions  and/or  recommendations  on 
programs  involving  advanced  technology,  Decisions  may  be  required  from 
the  individual  engineer  or  a  group  of  engineers,  Frequently,  the  decisions 
must  be  made  in  a  minimum  of  lead  time, 

The  tremendous  advances  in  technology  have  precipitated  a  situation 
whers  very  few  research  programs  are  of  a  single  technical  discipline,  They 
are  usually  related  either  directly  or  indirectly  to  other  technical  disciplines 
and  cannot  be  treated  singularly,  A  research  program,  regardless  of  the 
number  of  technical  disciplines  involved,  is  an  effort  to  explore  and  deter¬ 
mine  the  unknown  and  because  of  the  unknowns  is  not  always  conducive  to 
rigorous  technical  evaluation  by  an  individual  or  quite  often  a  small  group, 
Certainly,  as  the  number  of  disciplines  increase,  the  more  complex  the 
evaluation  becomes. 

The  engineer,  no  matter  how  competent  he  may  be  in  one  discipline, 
often  finds  himself  making  decisions  intuitively  rather  than  by  rigorous 
analysis  of  technical  facts,  This  is  so  because  quite  often  he  does  not 
have  the  necessary  facta,  he  does  not  have  the  time;  or  he  does  not  have 
the  necessary  capability  in  many  disciplines,'  When  the  decisions  are  made 
intuitively,  they  are  shaded  and  toned  by  the  engineer's  biases,  preconceived 
notions,  and  past  experiences,  As  the  amount  of  information  increases  in 
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a  multi-disciplined  problem  so  do  his  vacillations  between  biases  and 
preconceptions  in  the  peaces?  of  rr'.»V’"g  a  derision.  This  condition  is 
accentuated  where  the  research  program  is  such  that  the  technical 
opinions  of  others  must  be  considered. 

Therefore,  what  is  needed  is  a  systematic  approach  to  the  problem, 
consideration  oi  as  many  technical  factors  which  may  affect  the  decision 
as  possible,  and  a  method  of  weighting  the  factors  and  quantifying  the 
opinions.  In  other  words,  a  set  of  rules  are  determined  and  followed 
systematically  until  a  decision  can  be  reached, 

The  authors  were  recently  involved  in  a  problem  of  making  a  deci- 
ion  and  recommendations  on  certain  research  programs.  The  purpose 
of  this  paper  is  to  present  the  approach  taken  and  the  use  of  statistics  in 
the  decision  making  process  for  an  actual  case.  None  of  the  government 
agencies  or  research  groups  are  identified  except  the  U.  S.  Army  Missile 
Command  since  the  information  is  for  government  program  planning. 

II.  BACKGROUND.  The  U.  S.  Army  Missile  Command  (USAMICOM) 
is  the  technical  director  of  a  research  program  being  performed  by  a 
research  group  for  the  U.  S.  government.  This  research  program  was 
a  multi-disciplined  program  in  missile  phenomenology  involving  theory 
and  experimentation  in  such  disciplines  as  electromagnetics,  optics,  plasma 
diagnostics,  microwave-plasma  interactions,  aerothermochemistry , 
thermodynamics,  fluid  dynamics,  experimental  techniques  and  instrumen¬ 
tation.  This  program  was  one  of  several  similar  programs  of  an  overall 
research  program. 

The  group  directed  by  USAMICOM  (identified  as  Establishment  7) 
proposed  the  development  and  utilization  of  a  larger,  much  imjyroved 
hypervelocity  launcher  of  projectiles  for  research  purposes.  This  among 
other  things  precipitated  a  review  of  overall  research  effort  in  missile 
phenomenology.  In  view  of  this,  USAMICOM  was  requested  to  give 
recommendations  on  the  following  categorical  questions: 

1.  The  past  and  future  utilization  of  Establishment  7, 

II..  The  need  for  a  large  caliber,  light  gas  gun  and  possible 
uses  in  missile  phenomenology  research, 

III.  The  desirability  of  building  such  a  gun  at  some  establishment 
othe  r  than  7 . 
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The  experimental  approach  taken  by  the  authors  is  included  except  for 
tne  coding  of  ail  agendo*  and  research  group* 

ID.  THE  EXPERIMENT. 


A.  Design  Approach 

The  purpose  of  this  effort  is  to  provide  recommendations  in  three 
categories  which  are  of  concern  to  missile  phenomenology  research  programs . 

The  three  categories  are  as  follows: 

Category  I;  The  past  and  future  utilization  of  Establishment  7. 

Category  II:  The  need  for  a  large  caliber,  light-gas  gun  in 
missile  phenomenology  research. 

Category  III:  The  desirability  of  building  such  a  gun  at  some 
other  establishment. 

Due  to  USAMICOM'e  close  association  with  past  programs  and  in  an 
effort  to  carry  out  this  task  with  minimum  bias  and  maximum  objectivity, 
it  was  considered  appropriate  to  conduct  a  taclmical  survey  of  theoretical 
and  experimental  groups  associated  with  such  programs.  Time  limitations 
permitted  only  s  representative  sample  of  such  groups.  These  groups  are 
known  to  have  knowledge  pertinent  to  all  of  the  above  categories. 

It  was  anticipated  that  a  wide  variation  of  data  and  opinions  would  be 
obtained  from  these  groups  making  orderly,  efficient,  and  unbiased  analysis 
of  the  survey  results  difficult.  It  was  decided  that  a  method  of  analysis 
based  on  quantifying  of  data  and  opinions  must  be  used.  The  method  selec¬ 
ted  ie  the  "Case  Institute  Method  of  weighting  objectives"  and  is  described 
in  Reference  1  in  detail. 

It  was  decided  to  send  four  engineers  as  interviewers  to  visit  the 
selected  theoretical  and  experimental  groups.  The  groups  were  selected 
as  a  representative  cross  section  of  those  familiar  with  aeroballistic  range 
techniques  and  associated  research  programs,  and  therefore  able  to  contrib¬ 
ute  to  the  resolution  of  the  three  categorical  problems.  The  groups  were 
allowed  to  comment  on  or  off  the  record  to  increase  responsiveness. 
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The  establishments  were  visited  as  shown  in  Table  1.  It  can  be  seen 
that  Interviewer  1  visited  Establishments  2,  5,  7,  and  11;  Interviewer  2 
Establishments  3,  4,  9.  and  10;  Interviewer  3  Establishments  1  and  8; 
and  Interviewer  4  Establishments  6,  12,  and  13. 

For  consistency  of  the  interviews,  a  master  list  of  questions  consid¬ 
ered  pertinent  to  the  categories  was  provided  to  each  Interviewer  and 
discussed  at  each  establishment.  The  interviewers  recorded  a  summary 
of  facility  data  and  opinions  for  use  during  rating  of  the  factors,  Thereby, 
each  interviewer  obtained  sufficient  technical  background  information  upon 
whichhe  could  quantitatively  rate  ten  factors  considered  pertinent  to  each 
category.  The  ten  rating  factors  for  Categories  I,  II,  and  III  are  shown  in 
Tables  2,  3,  and  4  respectively.  The  ten  factors  were  selected  as  a  repre¬ 
sentative  sample  which  were  required  to  make  a  systematic  evaluation  of 
each  c&tego^y. 

The  ten, factors  in  Category  I  were  designed  to  rats  Establishment  7 
against  other  establishments,  The  establishments  chosen  for  7  to  be  rated 
against  were  1,  4,  6,  9,  12,  and  13,  These  represented  establishments 
similar  to  7  and  operated  by  all  government  agencies  of  the  Department  of 
Defense,  private  corporate  facilities  and  an  educational  institution, 

The  ten  factors  in  Category  III  were  designed  to  rate  establishments 
1,  4,  6,  9,  12,  and  13  against  7, 

The  ten  factors  in  Category  XI  were  designed  to  rate  the  opinion*  of 
both  theoretical  and/or  experimental  groups  on  the  need  for  a  large  light 
gae  gun. 

Each  interviewer,  after  discussion  of  the  factor  with  the  principle 
investigatory,  numerically  rated  each  factor  in  each  category  for  the 
establishments  visited,  These  ratings  were  between  0  and  4.  In  the  selec¬ 
tion  of  a  quantitative  rating,  if  the  rating  was  not  clearly  and  easily 
differentiated  from  the  mean  value  of  2,  the  rating  was  established  at  that 
level,  This  procedure  tende  to  minimise  individual  biae  and  enablee  the 
eurvey  to  approach  a  truly  unbiased  conclusion, 

B,  Factor  Rating  Criteria 


The  discussion  is  confined  to  the  typee  of  information,  data  and  com¬ 
ment!  obtained  for  use  ae  a  baeie  for  rating  the  ten  factor!  of  each  category. 


Category 


In  Category  I  the  first  factor  w&i  rated  on  the  basis  of  the  information 
received  on  program  objectives,  types  of  models  required,  instruments 
required,  and  types  of  data  collected.  Also  considered  was  reporting  in  ■ 
journals  or  at  symposiums,  the  opinion  of  tho  reporting  by  other  groups, 
and  the  degree  of  success  of  the  program.  The  rating  of  the  secondfaotor 
was  based  on  the  overall  instrumentation  capability  in  flow  field  visualisa¬ 
tion,  optical  radiation,  and  microwave  diagnostic  instruments,  as  well  as 
special  instrumentation,  The  third  factor  was  rated  on  such  criteria  as  ,  . 
complexity  of  model  shapes,  velocity,  and  data  gathering  and  launching 
problems.  The  fourth  factor  was  rated  on  the  basis  of  type  of  gun,  launch  ... 
weights,  velocities,  repeatability,  and  freedom  from  malfunction.  .  The  , 
fifth  factor  was  rated  on  the  basis  of  comments  of  professionals  who  have. 
had  close  or  personal  contact  with  professionals  of  Establishment  7,,  The 
sixth  factor  was  rated  on  the  basis  of  the  number  of  available  range*,  guns, 
standard  and  special  instruments,  and  utilisation  factor  of  the  facilities, 

A  criteria  of  minor  consideration  was  estimated  capital  investment.  The 
seventh  factor  was  rated  on  a  basis  of  some  of  the  same  criteria  as  factor  '' 
six  plus  the  ability  to  Initiate  programs  of  widely  varying  experimental 
parameters  on  short  notice.  The  eighth  factor  was  rated  oil  a  basts  of 
such  things  as  available  space,  facility  cooperativeneas,  and  facility  work¬ 
loads.  Most  establishments  have  existing  funded  programs  planned  and 
limited  staff  level  responsiveness.  The  ninth  factor  was  rated  on  relative 
defense  efforts  of  the  establishment.  The  tenth  factor  was  included  on  the 
premise  that  accomplishments  are  often  proportional  to  support  received. 


Category  II 


In  this  category  an  attempt  was  made  during  the  survey  to  establish 
the  need  for  a  large  caliber  gun  in  missile  phenomenology  research  and 
to  define  a  large  caliber  gun.  In  regard  to  the  large  gun  proposed 
reactions  varied  from  "it  is  feasible"  to  "lit  can't  be  done".  Others  etated 
a  preference  for  approaching  the  possibilities  of  designing  such  a  gun  in 
small  diameter  phases,  e.  g,  ,  2.  5  in,  ,  4  in.  ,  then  perhaps  6  in,  It  appears 
from  comments  obtained  that  a  3  or  4 -inch  gun  may  be  the  optimum  sice. 

A  4-inch  gun  capable  of  velocities  of  25,  000  feet  per  second  would  be  a 
sice  large  enough  to  allow  for  expansion  of  the  types  of  experiments  which 
could  be  performed  on  an  aeroballistic  range.  A  4-inch  gun  would  also 
be  more  easily  fabricated,  handled,  opened,  maintained  and  be  capable 
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a  secondary  issue,  the  prime  factor  being  the  determination  of  the  real 
need  for  a  large  caliber  gun,  Factnr*  one  and  tv>o  were  rated  on  the  basis 
of  the  capibility  of  a  large  bore  gun  to  expand  the  type*  of  experiment*  ‘ \ 
and  measurements  that  may  be  effectively  executed  under  simulated 
conditions,  These  factors  were  most  heavily  weighted  in  Category  H . 

The  concensus  is  that  this  is  the  foremost  justification  for  u  large  gun. 
However,  those  who  expressed  this  opinion  could  suggest  few  program* 
but  tome  example*  are:  (1)  launching  complex  geometrical  shapes, 

(2)  blast  vulnerability  studies,  and  (3)  on-board-model  telemetry  measure¬ 
ments.  The  fact  that  new  programs  cannot  currently  be  suggested  does 
not  exclude  many  suggestions  whsn  such  a  dsvice  is  available.  New  type* 
of  measurement*  will  be  developed  in  parallel  with  new  types  of  experi¬ 
ments  with  larger  models.  Factor  three  was  only  a  rating  of  the  opinion* 
of  the  interviewers  on  the  need  for  a  large  bore  gun,  These  opinions 
vary  strongly  from  favor  to  disfavor  and  are  reflected  in  column  3  of 
Table  6.  The  Xj  column  reflects  the  composite  of  all  factors  for  **ch 

establishment.  Factor*  four  and  five  sought  to  determine  if,  in  the  Opin¬ 
ions  of  other*,  larger  model*  would  improve  th*  thresholds  of  measure¬ 
ments  made  by  current  instruments  at  a  given  simulated  altitude  or  provide 
equal  thresholds  at  a  higher  simulated  altitude.  Some  respondents  indi¬ 
cated  that,  on  a  quantitative  analysis,  significant  improvement*  would 
be  obtained,  Other  respondents  feel  that  larger  guns  would  Improve 
thresholds  and  resolution  significantly,  especially  in  optical  measurements 
but  not  on  microwave  measurements.  Respondents  generally  agree  that 
simulated  data  can  be  more  easily  utilised  in  theoretical  modeling  and 
computation*  than  in  full  seal*.  Some  respondents  did  not  fesl  that  this 
was  particularly  true  to  the  point  of  Justifying  a  larger  gun  than 'is 
nominally  used,  e.  g.  ,  1-1/2  inch  gun.  Some  of  the  respondents  to  factor 
seven  could  not  comment,  especially  if  this  factor  is  viswsd  from  ths 
standpoint  of  a  large  gun  reliability,  capital  cost,  and  useful  life.  Other 
respondents,  even  in  view  of  these  criteria,  fesl  that  more  usable  data 
can  be  obtained  at  lees  expense  on  ballistic  ranges  than  under  full  scale 
condition*,  The  overall  response  to  factor*  eight  and  nine  varied  from 
neutral  on  eight  to  *lightly  negative  on  nine.  One  respondent  described 
quantitatively  that  examinations  of  scaling  limit  increase*  show  that  from 
10,000  to  20,000  feet  of  altitude  may  be  obtained  by  a  fivefold  increase 
in  site  for  binary  scaling  of  waks  electron  densities.'  Also,  only  a  20 
percent  increase  in  wake  lengths  that  could  be  scaled  would  be  obtained. 
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Factor  ten  wu  included,  at  very  low  weight,  merely  to  emphasise  this, 
advantage  of  ballistic  range  data  gathering  when  contrasted  to  full- scale 
data,  While  full  scale  does  represent  the  real  case,  for  purposes  pf 
study  repeatability  is  highly  desirable,  In  view  of  the  fact  that  such 
diverse  opinions  and  wide  variations  ii.  responses  were  obtained,  the 
analysis  was  made  easier  by  use  of  the  Cass  Institute  Method  approach. 

Category  Ill 

This  category  assumes  that  a  large  caliber  gun  is  needed.  It  is, 
therefore,  important  to  determine  the  best  places  that  such  a  device 
Bhould  be  installed  and  operated. 

The  installation  of  a  large  caliber  gun,  which  would  be  heavy,  long,,, 
and  cumbersome,  would  require  that  the  establishment  have  the  neceisary 
heavy  moving  equipment,  transfer  locations,  and  housing  to  properly  : 
operate  and  maintain  it,  Factor  one  considers  these  present  capabilities 
without  new  construction, 

The  Installation  of  a  large  caliber  gun  would  necessitate  increasing 
ths  number  of  persons  required  to  operate  and  maintain  it  in  a  data  - 
gathering  program.  The  operation  and  maintenance  necessitates  handling 
and  storage  of  large  amounts  of  munitions  and  H2  of  He  , gas,  fabrication 

of  larger  models  and  sabots,  telementry  packages,  and  other  incidental 
items  required  to  effectively  pursue  such  a  program,  In  establishments 
where  programs  are  presently  funded  to  accomplish  a  mission,  such  a 
large  program  would  perhaps  overload  their  present  capability,  In  view 
of  this,  ths  desire  of  an  establishment  to  participate  in  a  program 
utilising  a  large  caliber  gun  is  important.  This,  in  turn,  is  a  function 
of  their  interests  in  the  experimental  programs  to  be  pursued  with  a 
large  caliber  gun. 

The  ratio  of  chamber  diameter  to  model  diameter  for  good  compati¬ 
bility  has  been  estimated  between  20  and  30,  Therefore,  a  5-inch  model 
would  require  (taking  the  average)  a  chamber  of  125  inches  (approximately 
10  feet),  Some  establishments  would  require  additional  chambers  for 
4-or  5-inch  models  if  this  ratio  is  accepted,  Therefore,  some  establish¬ 
ments  may  have  the  desire  and  Interest  but  not  adequate  facility  and 
personnel  capability  or  range  compatibility. 
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Other  important  considerations  are  the  attitude  of  the  establishment 
to  the  full  -  or  part-time  participation  of  contractors  in  data  gathering 
on  the  range  and  the  participation  of  contractors  intermittently  to  obtain 
a  few  data  points  of  a  specific  interest.  This  requires  that  a  certain 
amount  of  space  on  the  range  for  instrumentation  be  available.  Quite  often 
the  data  can  be  gathered  on  shots  of  opportunity. 

In  anticipation  of  research  contractor  participation,  the  accessibility 
of  the  facility  is  important  to  maximum  utilization  of  the  facility.  In 
conjunction  with  this  will  be  the  ability  to  control  and  direct  programs 
and  program  changes.  Program  orientation  is  also  important.  It  may  be 
desired  to  pursue  a  basic  long  term  program  with  short  specific  tasks 
overlaid,  the  results  of  which  may  on  occasion  change  the  basic  program 
orientation. 

Finally,  the  cost  of  a  large  gun  is  considered.  The  overall  opinion 
is  that  the  costs  will  probably  not  differ  greatly  between  government 
establishments.  However,  an  industrial  or  corporate  facility  may  be 
more  economical  than  the  government  facilities. 

C .  Numerical  Analysis 

The  Case  Institute  Method  of  weighting  objectives  (1)  was  selected  for 
use  in  weighting  the  factors  and  quantifying  the  respondent's  comments 
and  opinions. 

The  lack  of  a  universal  standard  deviation  and  the  small  sample 
dictated  the  use  of  Student's  't '  distribution  for  test  of  significance  of 
the  results. 

In  the  Case  Institute  Method,  the  ten  factors  are  weighted  as  follows: 

1.  One  factor  in  each  category  is  rated  most  important  and  given 
a  value  of  1.  00.  Each  of  the  other  nine  factors  are  then  rated  between  0 
and  1.  00  according  to  its  relatively  judged  importance. 

2.  After  all  factors  in  a  category  are  rated,  the  most  important 
factor  is  compared  to  the  other  nine  collectively  as  to  importance  in  the 
category.  If  it  is  judged  more  important  than  the  other  nine  collectively, 
the  value  of  1.  00  first  assigned  is  changed  to  a  value  larger  than  the  sum 
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of  the  other  nine  values.  If  the  most  important  factor  ia  considered  'o  be 
of  the  same  importance  aa  the  other  nine,  the  value  for  the  moat  important 
factor  ahould  be  equal  to  the  sum  of  the  other  nine  factors.  If  it  is 
conaicierec  to  be  of  importance  than  all  th»  nine,  than  its  value 

is  adjusted  to  some  value  less  than  the  sum  of  the  other  nine. 

3.  The  most  important  factor  and  its  weight  are  established. 
Next, the  second  most  important  factor  is  compared  to  the  remaining 
eight.  Its  weight  is  established  in  the  same  manner  ae  described  in  2 
above.  When  the  factor1*  weight  is  established,  the  procedure  continue* 
to  the  third,  fourth,  etc,  most  important  factor  until  all  10  factors  are 
weighted. 

4.  This  procedure  is  followed  for  all  three  categories. 

A  composite  of  the  weighting  for  all  categories  is  shown  in  Table  8 
in  order  of  descending  weight.  The  factors  for  all  three  categories  can 
be  seen  in  Tables  2,  3,  and  4, 

The  method  of  rating  the  factors  was  to  ul*  the  five  discrete, numer¬ 
ical  levels  0,  1,  2,  3,  4.  In  Category  I,  each  establishment  contrasted 
with  7  was  sst  at  levsl  2  and  7  ratsd  below  or  abovs  at  0  or  1  and  3  or  4 
respectively.  In  Category  II,  a  neutral  position  on  each  factor  by;  the 
respondent  was  sst  at  2  and  the  degree  of  disfavor  or  favor  of  Category 
II  at  0  or  1  and  3  or  4  rsapectively.  In  Category  III,  7  wee  eat  at  2  for 
each  factor,  and  each  establishment  was  rated  below  or  above  with  0  or 
1  and  3  or  4  respectively. 

The  rating  established  for  each  factor  in  each  category  was  multiplied 
by  the  corresponding  factor  weight  and  is  recorded  in  Tables  5,  6,  and  7, 
The  values  are  summed  for  each  establishment.  In  order  to  normalise 
the  range  of  response  for  each  establishment  In  each  category,  the  follow¬ 
ing  equation  is  used; 

v  a  £lll£*2£  wt  *  factor  rating)  -  2  x  Z (factor  wt) 
i  "  i  x  Z  (factor  wt) 

For  Category  Is 

Z(w  x  R  )  -  2  x  3.12 

x  «  - -rA-TS -  * 

i  2  x  3. 12 
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For  Category  11‘. 


Xi  ' 


E(W,x  Rf)  -  2  x  3,?5  ^ 
-  2  x  3.7  5 


For  Category  III'. 

E(W{  x  Rf)  ■  2  x  1.49 


Xi  = 


2  x  1.  49 


For  all  categories 

Th.  limit,  tor  o.ch  X,  in  .11  e.t.go.lf  b.com.. 


for 


R{ -  ° 
Rf-4 
Rf  •  2  X£ 


X 


s  -1 
.  +1 

■  0  ■  X*  (hypothe.ie  value) 


where 

X£  ■  «»tabli»hment  computed  reeponie 
Wf  ■  factor  weight 
r£  ■  factor  rating 
M  .  number  of  eetabliehmente. 

The  .ample  deviation  (S)  for  each  category  ii 

’e(x1  -  K}2  "j 
S  "  [-tt-  .  ’ 
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t 


X  -  X 

■  s77n 


Using  the  data  from  Tables  5,  6,  and  7  for  Categories  I,  12,  and 
Ill  respectively,  we  calculate  the  sample  standard  deviations! 


* 

«  0.149 


Before  the  't '  teats  are  made,  a  confidence  level  of  70  percent  is 
set,  which  is  considered  appropriate  for  research  (i,  a. ,  risk  otiirst,. 
kind’*  a  *  ;  30)  and  the  following  hypotheses  are  made  on  each  category! 

Category  1;  There  is  ho  significant  difference  in  utilia- 
tion  of  7  and  other  establishments  (i.  e.  , 
t*  ■  0). 


Category  Hi  There  is  no  significant  need  for  a  larger  caliber 
gun  in  the  missile  phenomenology  research 
program  (i.  a.  ,  |u  ■  0). 

Category  lilt  There  is  no  significant  difference  between 
establishment*  where  a  large  gun  should  be 
built  (i.  a, ,  |i  »  0) , 


The  't *  tests  are  computed  for  each  category; 


t 


I 


■  067  -0 
.149//*' 


1.105 


*The  risk  of  rejecting  a  hypothesis  when  it  is  true,  Also  celled  the 
producer's  risk, 


This  task  ia  one  which  is  highly  complex.  Many  technical  areaa  of 
an  advanced  nature  are  involved,  An  honest  and  sincere  effort  has 
been  made  to  reach  an  unbiased  and  technically  sound  solution,  The 
groups  queried  have  provided  comments  which  are  spontaneous  and 
which  instinctively  draw  on  years  of  technical  experience  pertinent  to 


4 

tII 

”787713  =  1,41 
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*111  = 

.256/76  *  2‘05 

* 

The  computed  values  are  compared  with  Student's  't'  table  values 
as  shown  below: 

Table  Value 

Computed  Value 

Degrees  of 

Percentile  Point 

for  Categories 

Freedom 

70  80  90  95 

I 

«  1,105 

5 

0.  56 

0.  92 

1.48 

2.01 

II 

■  1.41 

12 

0.  54 

0.87 

1,  36 

1.  78 

Ill 

-  2.  05 

5 

0.  56 

0.  92 

1.48 

2. 01 

It  can  be  seen  that  the  tests  for  sill  three  categories  are  significant 
at  the  original  level  of  confidence  of  70  percent  which  is  considered 
appropriate  for  advanced  research  projects.  As  the  tests  are  significant 
at  this  level  (the  computed  value  is  greeter  than  the  table  value),  all 
three  hypotheses  are  rejected.  The  highest  level  at  which  the  tests  are 
significant  and  the  hypotheses  rejected  are  Category  1*80  percent, 
Category  11*90  percent,  Category  III  a  95  percent. 

V 

If,  however,  it  is  considered  that  the  level  of  confidence  should  be 
95  percent,  then  the  tests  for  ..Categories  I  and  II  are  not  significant  and 
the  hypotheses  accepted,  Category  III  i‘s  still  significant  but  inconsequen 
tial.  For  the  purposes  of  decision  making  in  this  type  research  and 
development  programs,  the  95-percent  level  of  confidence  is  considered 
excessively  high  by  the  Investigator. 

D.  Summary  and  Conclusions 
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the  problem,  Therefore,  .conaid*Mlr>l» '  JgA.111  ittsntisr.  and  ticlu*ic*I 

capability  have  been  concentrated  on  the  tKr.ee  categories.  It  is  not 
supposed  or  proposed  that  every  facet  has  been  considered  and  explored, 
nor  has  a  rigorously  technical  approach  been  used  as  this  would  be  a 
formidable  task.  However,  a  representative  sample  of  the  foremost  :■ 
factors  has  been  considered,  and  the  technical  analysis  was  performed 
mentally  by  the  respondents, 

A  systematic  approach  to  the  analysis  of  a  highly  complex  problem 
has  been  used  as  shown  in  the  numerical  analysis.  The  importance  of 
this  approach  is  the  capability  to  make  a  decieion  in  the  realm  of  uncer¬ 
tainty  and  random  variation.  ••  . 


Review  of  the  rssults  of  the  ratings  of  Category  I  presented  in 
Table  5  shows  that  (considering  all  factors)  7  rates  below  9  at  >0,178 
(or  17. 8%)  and  slightly  above  all  others  with  4  and  13  cloeest  with  a  . 
+0.008  or  (0.80%)  and  +0.024  (or  2.4%),  respectively.  Comparing ‘7 
to  all  other  establishments  for  all  factors  7  rated  at  +0.0665  (6.65%) 
which  ia  significant  when  compared  to  the  eample  standard  deviation  by 
the  't1  test. 

Review  of  the  results  of  the  ratings  of  all  factors  for  Category  IZ, 
presented  in  Table  6,  shows  that  2  was  strongly  not  in  favor  of  a  large 
gun  by  a  value  of  0,  701,  followed  by  10  and  8.  Seven  was  strongly  in 
favor  of  a  large  gun  with  a  value  of  +0.  948,  followed  by  5,  4,  6,  and  9. 
Twelve  and  1  were  slightly  in  favor,  with  values  of  +0.  040  and  +0,  041, 
respectively.  On  an  overall  comparison  of  all  factors  and  all  establish¬ 
ments  there  was  a  favorable  response  of  +0.148  (14,  8%),  This  evaluation 
does  not  Include  the  exact  launch  tube  diameters. 

Review  of  the  results  of  rating  the  factors  in  Catsgory  ID,  presented 
in  Table  7,  shows  that  9  with  a  value  of  -0,  0067  and  13  with  a  value  of 
-0.  0436  compare  closest  with  7  as  the  place  to  build  a  large  gun.  Twelve 
wae  least  favorable  with  a  value  of  -0.  711. 

Therefore,  on  the  basic  of  the  analysis  of  the  overall  results  shown 
and  within  the  limits  of  this  study  ths  following  conclusions  wsre  drawn* 

Category  I 

There  is  an  apparent  difference  in  the  overall  usefulness  of  7  com¬ 
pared  to  other  facilities,  Thers  is  a  significantly  positive  opinion  that 
7  may  be  effectively  utilised  in  the  future. 
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Category  II 

There  is  an  apparent  need  for  a  large  caliber  gun  in  the  missile  phe¬ 
nomenology  research  program.  There  is  a  significantly  positive  opinion 
that  such  a  device  is  needed  presently  and  in  the  future. 

Category  III 

There  is  an  apparent  difference  between  establishments  where  a 
large  gun  could  be  built  and  utilized.  Establishment  7  is  a  foremost 
contender  as  a  desirable  establishment  for  developing  the  large  caliber 
gun.  Recommendations  on  program  continuation  together  with  suggested 
experiments  were  made  based  on  these  conclusions. 

IV,  DISCUSSION.  The  preceding  case  la  a  real-world  example  of 
how  operation  research  and  statistical  methods  can  be  utilized  to  assist 
in  the  process  of  making  technical  decisions.  The  particular  features  of 
this  approach  are: 

1.  An  inter-disciplinary  team  is  utilized  to  bring  a  variety 
of  technical  viewpoints  to  bear  upon  the  problem. 

2.  The  results  of  such  a  team  effort  are  quantified  to  make 
it  possible  for  analysis  to  be  made  at  optimum  objectivity. 

3.  Statistical  techniques  are  applied  to  evaluate  the  quantified 

results. 

The  key  feature  of  such  methods  is  the  concept  of  risk  and  proba¬ 
bilistic  conditions.  Such  an  approach  is  particularly  useful  in  the  realm 
of  decision-making  since  the  risks  are  often  great  and  the  probabilistic 
environment  is  every  present.  Under  such  conditions  there  is  no 
opportunity  for  drawing  a  definite  conclusion.  A  decision  can  only  be 
made  at  a  given  lev^l  of  confidence.  The  risk  of  a  decision  being  wrong 
becomes  a  calculated  part  of  the  problem. 

Tbe  use  of  quantitative  methods  for  expressing  the  results  of  the 
experiment  can  often  lead  to  a  process  of  over  interpretation  of  results 
often  to  the  neglect  of  sound  technical  judgement.  Obviously,  the 
decision  cannot  be  made  solely  with  such  methods.  At  best,  the  decision¬ 
maker  can  be  fortified  with  certain  analyses  of  the  experimental  results 
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that  will  provide  a  statement  of  the  risk  he  would  take  if  he  should  make  a 
decision  in  one  direction  or  another.  Such  factual  data  ce& .  often provided 
with  a  minimum  of  bias  from  lower  echelons  so  that  the  decieion-maker 
can  benefit  from  it  while  exercising  his  best  judgement  in  the  problem. 

The  experiment  was  basically  concerned  with  the  determination  of 
technical  facts  that  existed  within  each  of  the  installations.  To  obtain  auch 
facts  required  us  to  go  through  several  "bias  filters"  such  as*. 

1.  The  ability  and  willingness  of  the  installation  representative 
(the  interviewee)  to  stats  the  true  facts  that  exist  in  his  group  as  free  of 
bias  and  inaccuracies  as  possible. 

2.  The  ability  of  the  interviewer  to  gather  and  tranemlt  the  data 
to  the  investigator  with  a  minimum  of  his  own  personal  bias  involved. 

3.  The  ability  of  the  investigator  to  compile  the  final  data  as  ;  . 
free  of  his  own  personal  bias  as  possible. 

To  accomplish  tha  above  purposes  is  obviously  no  easy  task  under 
any  circumstances  The  problem  was  faced  in  the  investigation  by  utilising 
these  basic  techniques) 

1.  A  multiple  of  closely  related  questions  were  used  to  conduct 
the  interviewers  with  each  installation  representative. 

2.  The  interviewee  bias  was  observed  and  evaluated  by  the  inter¬ 
viewer  in  each  case. 

3.  The  data  was  transmitted  to  the  investigator  and  a  concerted 
attempt  was  made  on  the  part  of  the  investigator  to  balance  the  bias  of  the 
interviewer  and  interviewee  through  the  conduct  of  an  extensive  "debrief* 
ing"  procedure. 

4.  The  bias  of  the  investigator  waa  controlled  by  both  the  influence 
of  the  interviewer  in  the  debriefing  session!  and  the  systematised  method 

of  quantifying  the  reeults. 

Obviously,  the  effort*  juet  described  could  never  hope  to  eliminate 
all  bias  and  inconsistencies,  The  recognition  of  this  fact  leads  us  to 
evaluate  the  final  results  with  techniques  that  have  been  developed  for  such 
situations. 


We  have,  in  effect,  produced  quantified  result*  within  an  enviromeat 
of  uncertainty,  Such  uncertainty  is  made  up  of  two  basic  elements,  Thfct 
is,  the  observed  differences  in  results  between  installations  can  be 
attributed  to: 

1,  Differences  that  are  explained  by  residual  errors  and  biases 
that  etill  remain  in  the  experiment  in  spite  of  the  proceduree  that  were 
established  to  eliminate  them, 

2,  Differences  that  are  explained  by  real  effects  of  the  installa¬ 
tion  on  the  category  in  question  as  far  as  the  study  can  determine. 

The  test  of  hypothesis  used  in  the  analysis  served  to  partition  these 
two  basic  causes  of  observed  differences,  To  say  that  a  resulting  effect 
was  significant  is  to  say  that,  within  th#  limits  of  this  investigation,  the 
observed  differences  between  the  selected  installations  cannot  be 
attributed  merely  to  experimental  error.  The  conclusion  is  therefore 
drawn  that  a  real  difference  exiete  and  a  positive  conclusion  is  therefore 
drawn.  It  is  important  to  not*  that  for  each  conclusion  there  is  a  compa¬ 
rable  level  of  confidence.  Within  th#  realm  of  an  environment  of  uncer¬ 
tainty  all  conclusions  or  decisions  mutt  carry  this  element  of  risk. 

V.  REFERENCE 

1.  Churchman,  Ackoff,  and  Arnoff,  "Introduction  to  Operation  Research,  " 
John  Wiley  and  Son,  New  York,  New  York. 
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Establishment  Nr,  7  Utilisation  Evaluation  Factore 


Category  I 

Factors  are  listed  in  descending  order  of  established  weights. 

Each  factor  rated  with  2  representing  each  establishment  against  which 
Establishment  7  is  rated.  The  rating  levels  are  chosen  by  this  interviews 
and  the  chairman  of  the  survey  committee. 

1.  How  do  7 ' s  past  program  results  compare  to  other  establishments  ? 

2.  How  does  7's  past  instrumentation  rate  in  comparison  to  other 
establishments  7 

3.  How  did  7's  program  rats  with  other  ranges  in  degrse  of  difficulty 
to  perform? 

4.  How  does  7's  past  gun  performance  rate  in  comparison  to  other 

range • ? 

5.  How  do  7 '  •  professionals  compare  with  professionals  of  other  ranges? 

6.  How  does  7's  past  facility  development  rate  in  comparison  to  other 
rangoo  ? 

7.  How  does  7's  utility  as  r.  data  gathering  facility  in  future  compare 
with  other  range*  ? 

8.  How  doe*  future  possibility  of  contractors  participation  on  ranges 
at  7  compare  to  other  establishments  ? 

9.  How  strong  is  7's  desire  to  continue  participation  in  missile 
phenomenology  research  compared  with  other  ranges? 


10.  How  dose  7 '  a  past  funding  compare  to  other  range  program*? 
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TABLE  3 

Large  Borg  Gun  Evaluation  Factors 
Category  II 


Factor*  are  lieted  in  descending  order  of  eetabliehed  weight*, 
Each  factor  rated  at  level*  between  0,  1,  2,  3,  and  4  on  baeia  of  d*t*  > 
and  oplnione  gathered  with  2  representing  neutral  opinion.  The  rating 
level*  are  choaen  by  the  interviewer  and  the  chairman  of  the  aurvey 
committee. 

1.  Will  they  expand  the  type*  of  experiment  that  may  be  effectively  • 
executed  under  aimulated  condition*  ? 

2.  Will  they  open  avenue*  of  new  type*  of  measurements? 

3.  What  1*  opinion  of  other*  doing  theoretical  work  on  need  for  large 
bore  gun*  ? 

4.  Will  they  increaae  observable*  level*  at  higher  limulated  altitude* 
aignificantly  ? 

5.  Will  larger  bore  gun*  improve  reliability  and  confidence  in  range 
measurement*? 

6.  What  1*  opinion  of  other*  on  the  value  of  aimulated  data  v£  full 
scale  for  utilisation  in  theoretical  modeling  and  computation*  T 

7.  How  doe*  coat  of  uaable  ballistic  ring*  data  gathering  compare 
with  uaable  full  acale  data  gathering  ? 

8.  Will  they  contribute  aignificantly  to  acaling  between  theory  and 
full  scale  ? 

9.  Will  they  contribute  significantly  to  the  establishment  of  binary 
scaling  limits  ? 

10.  What  is  the  opinion  of  ballistic  range  data  gathering  capability 
from  standpoint  of  rapsatability  ? 


172 


TABLE  4 


Large  Bore  Gun  Location  Evaluation  Factor ■ 
Category  III 

Factors  are  listed  in  descending  order  of  established  weights, 

Each  factor  rated  at  levels  between  0,  1,  2,  3,  and  4  on  basis  of  data 
and  opinions  gathered  with  2  representing  Establishment  7  against  which 
each  establishment  is  evaluated,  The  rating  levels  are  chosen  by  the 
interviewer  and  the  chairman  of  the  survey  committee, 

1.  To  what  degree  are  other  establishments  abls  to  accommodate  a 

large  gun  from  standpoint  of  housing,  operating,  and  maintenance 
without  facility  construction  relative  to  7  ?  •  ■>  ■;  •  ■ 

2.  What  was  capability  of  othar  establishments  for  taking  on  additional 
range  measurements  programs  relative  to  7  ? 

3.  How  strongly  do  other  establishments  indicats  they  want  to  build  •v” 
a  large  bore  gun  relative  to  7  ? 

4.  What  was  Interest  of  other  establishments  in  taking  additional 
programs  relative  to  7  ? 

5.  To  what  degree  is  thair  present  range  chamber  diameter  compatible 
with  large  models  relative  to  7  7 

6.  What  is  attitude  of  other  establishments  toward  contractor  participa- 
tion  in  data  gathering  on  thair  ranges  relative  to  7? 

7.  la  space  presently  mors  available  on  their  ranges  for  contractors' 
utilisation  relative  to  7  ? 

8.  How  does  accessibility  of  other  establishments  compare  to  7? 

9.  How  does  the  ability  to  control  programs  at  other  establishments 
compare  to  7  ? 

10,  How  will  cost  of  large  gun  development  at  other  establishments 
compare  to  7  ? 


TABLE  5.  CATEGORY  I  FACTOR  RATING 


be  desl 


TABLE  6.  CATEGORY  II  FACTOR  RATING 
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2 
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m 
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0.30 
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0.92 

0.74 
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£ 
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1.49 

IMPROVEMENT  CURVES;  PRINCIPLES  AND  PRACTICES 


Jerome  H.  N,  Selman,  Steven*  Institute  of  Technology, 
Rep.  the  U.  S.  Army  Munitions  Command,  Dover,  N.  J. 


To  build  the  1000th  B-29  Aircraft  took  only  3%  of  the  time  required  to 
build  the  first.  To  build  your  first  window  screen  or  dog  house  will  take 
you  more  time  them  each  succeeding  one --unless  you  are  a  professional 
window  screen  or  dog-house  maker.  This  feeling  i«  intuitive.  The  estima¬ 
tion  of  time  reduction  for  each  succeeding  item,  based  upon  judgment  and 
expeiience,  is  attributed  to  a  human  "learning"  effect,  Mathematically, 
the  way  to  express  this  condition  would  be  to  use  a  re  duct  ion -type  function: 

A  straight  line  equation  with  constant  negative  slope  for  a  constant  linear 
reduction  of  cost  with  quantity;  a  hyperbolic  equation  with  negative  exponent 
for  rapid  initial  reduction  of  coat  with  quantity,  then  slowing  down  to  a 
limit;  more  complex  equations  which  are  designed  to  reflect  the  phases 
of  the  specific  learning  situation. 

Models  of  the  cost-quantity  relationship,  as  a  predictive  technique, 
came  into  general  use  in  the  airframe  industry  during  World  War  II  after 
their  development  in  the  1930's,  T,  P.  Wright 'e  pathfinding  article* 
hyperbolically  related  the  average  direct  man-hour  coet  to  the  number  of 
airframee  produced.  Others  have  modified  Wright's  model  to  show  the 
inverse  relationship  between  the  direct  labor  hours  per  unit  versus  isntity 
produced;  this  latter  formulation  being  known  as  the  Unit  (improvement) 
Curve.  A  linear  improvement  curve  having  linear  component  curves 
implies  that  the  rate  of  learning  is  the  same;  intuitively,  again,  the 
assumption  of  constant  learning  rate  in  all  operations  is  open  to  question. 
Wright  was  of  the  opinion  that  different  rates  of  learning  are  found  in  the 
airframe  manufacturing  process,  but  he  did  not  inquire  into  the  implications. 

Studies  in  the  then-new  airframe  industry  for  sub-sonic,  reciprocating 
engine,  electrically  simple  aircraft  indicated  that  although  the  precentage 
slope  of  the  improvement  curve  varied,  for  every  doubling  of  successive 
quantities  of  aircraft,  the  percentage  value  was  a  constant  percentage  of 
the  unit  value  of  the  quantity  immediately  prior  to  doubling.  The  percentage 
reduction  was  approximately  80%,  This  meant  that  each  time  the  quantity 
was  doubled,  the  man-hours  required  to  make  that  designated  aircraft  was 
80%  of  the  man-hours  required  immediately  prior  to  doubling.  Plotting 
the  improvement  curve  on  logarithmic  grids  gives  a  "straight  line  curve", 


*T,  P.  Wright,  "Factors  Affecting  the  Cost  of  Airplanes,  11  Journal  of  the 
Aeronautical  Sciences.  Vol.  3,  February,  1936,  pp.  122-128. 
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as  the  grids  are  so  scaled  that  the  interval  between  doubled  quantities  are 
equal;  i.e.  ,  the  distance  between  one  and  two  is  the  same  as  the  distance 
between  two  and  four,  or  four  and  eight,  or  eight  and  sixteen,  etc. 

Of  course,  the  linear  hypothesis  should  be  discarded  whenever  the  unit 
curves  of  man-hours  and  cost  depart  significantly  from  linearity- -"signifi¬ 
cant  departure"  being  determined  from  the  slopes  of  the  parallel  linear 
component  curves,  based  on  the  error  permissible  in  the  problem  in  hand. 

Improvement  curves  are  expressed  in  terms  of  percentages,  such  as 
80%  Curve,  90%  Curve,  92%  Curve,  etc.  The  percentage  figure  referring 
to  the  fact  that  man-hours  tend  to  decrease  by  a  definite  amount  each  time 
the  quantity  produced  is  doubled,  By  correlation  and  other  statistical 
techniques  it  has  been  shown  that  a  graph  of  the  actual  performance  data 
(cost,  as  inferred  by  man-hours  per  unit  versus  quantity  produced,  or 
tasks  accomplished)  may  be  approximated  by  a  hyperbolic  function  of  the 
form  y=axb,  with  a  relatively  high  degree  of  significance.  The  fundamental 
hyperbolic  shape  is  postulated  rather  than  tested  (for  linearity  on  double¬ 
log  scales  versus  some  alternate  non-linear  functional  form  for  comparison), 
as  a  descriptive  device  for  accumulated  data,  In  Improvement  Curve 
terminology,  y,  is  in  direct  man-hour  coat,  a,  is  the  direct  man-hour 

cost  for  "unit  Number  one",  and  b  defines  the  "slope"  of  the  curve - 

"slope"  being  che  ratio  of  the  unit  (or  average)  man-hour  cost  at  two 
cumulative  outputs  that  differ  by  a  factor  of  two  (2),  so  that  the  slope  is 
2^.  Wright's  empirical  data  on  unspecified  aircraft  yielded  a  "b"-value 
of  -.  322,  giving  the  popular  "80%  Curve".  On  arithmetic  grid  the  80% 

Curve  with  a  unit  one  cost  of  1000  man-houre  is  shown  in  Figure  1,  the 

- .  322 

equation  being  y  =  lOOOx 

To  illustrate  the  mechanics  of  constructing  improvement  curves,  the 
80%  Curve  will  be  done  in  three  parts;  ae  shown  in  Figure  2  : 

The  Unit  Time  Line;  Given  a  value  for  any  unit  P  and  the  slope  of 
the  Improvement  Curve  in  percentage  form,  draw  a  line  from  point 
P  through  a  point  X  eo  that  it  will  be  twico  the  unit  number  of  P, 
i.e.,  P  equals  twice  X;  and  the  value  of  X  will  be  the  value  of  P, 
multiplied  by  the  percent  slope  of  the  curve,  Equation:  =  ax^ 

for  unit  curve,  1 

The  Average  Time  Dine  Per  Cumulative  Unit:  The  Cumulative 
Average  line  is  drawn  in  two  steps: 
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1.  The  Asymptote.  The  Cumulative  average  line  approaches  a 
straight  line  which  is  parallel  to  (after  about  the  15th  unit)  and 
higher  than  the  unit  line.  To  construct  the  asymptote,  obtain  the 
"b"  for  the  improvement  curve  in  question.  Draw  the  asymptote 
parallel  to  the  unit  line  so  that  the  values  of  all  points  on  the 
asymptote  are  equal  to  l/(l+b)  times  the  values  on  the  unit  line. 
For  the  80%  Curve,  the  conversion  factor  for  (l+b)  is  0.  687,  as 
given  on  Table  I,  giving  each  point  on  the  asymptote  a  value  of 
1.475  the  corresponding  value  on  the  unit  line.  Equation; 

=  a  Nb 

y  =  l+b”  ‘ 


2.  The  Cumulative  Average  Line.  As  an  approximation  for 
values  between  2  and  about  15,  the  cumulative  average  values 
for  any  unit  X  is  approximately  equal  to  the  value  shown  on  the 
asymptote  for,  X+3.  That  is,  the  average  cost  of  the  4th  unit 
is  approximately  equal  to  the  value  of  the  asymptote  at  unit  7. 

For  practical  purposes,  the  average  line  for  units  16  and  above 
may  be  considered  to  equal  the  values  of  the  asymptote.  Equation 


y 


n 

Mi 


The  Total  Line;  Draw  a  line  from  the  value  of  unit  number  one  to 
a  point  at,  say,  unit  number  10,  which  has  a  value  equal  to  10  times 
the  cumulative  average  value  of  unit  number  10.  It  is  logical  that 
the  total  time  for  the  first  ten  units  is  equal  to  10  times  the  average 
time  (cost)  of  the  first  ten  units.  Equation; 


the  corresponding  asymptote  is  N  times  the  cumulative  average 
asymptote,  just  as  the  Total  line  is  N  times  the  cumulative  average 
line . 


Improvement  Curves  have  been  utilized  in  the  Aerospace  Industries  for 
Cost  estimates,  scheduling,  efficiency  comparisons ,  procurement  and 
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subcontracting,  facilities  planning,  personnel  planning,  long-range  fore¬ 
casting,  etc,  ,  and  was  proposed  for  various  industries  such  as  home  appli¬ 
ances,  electronics,  construction,  machine  ,  ship  building,  etc.  The 

accuracy  of  the  Improvement  Curve  function  as  an  estimating  device  is 
dependent  upon  a  number  of  factors,  including: 

Accuracy  of  Basic  Estimate 

Choice  of  the  Improvement  Kate  exponent  "b" 

Non-linear  elementa  is  the  real  world 
Changes  in  the  output  rate 
Design  Changes  in  product 
Influx  of  "green"  manpower 
Exit  of  skilled  manpower, 

The  basic  tenet  of  Improvement  Curve  philosophy  is  where  there  it  life 
(people)  there  can  be  learning,  the  more  man-oriented  the  work,  the  more 
learning  potential  possible,  Figure  3  illustrates  the  generally  accepted 
improvement  curve  percentages  for  various  man-machines  mixes:  75% 
Man-25%  Machine  for  the  80%  Improvement  Curve;  50%  Man-50%  Machine 
for  about  85%  Improvement  Curves;  25%  Man-75%  Machine  for  the  90% 
Improvement  Curves,  etc, 

Munitions  Command  Regulation  715-1  requires  thorough  justification 
where  "program  costs  are  not  reduced  in  accordance  with  expected  learn¬ 
ing  curve  costing.  "  The  technique*  of  the  learning  or  improvement  curves 
can  set  realistic  management  goals  for  setting  expected  rates  of  improve¬ 
ment  in  reducing  operating  expensee  in  the  Army  "Five-Year  Cost  Reduction 
Program". 

Operations  develop  trends  that  are  characteristic  of  themselves,  Pro¬ 
jecting  such  established  trends  is  mcr  e  valid  than  assuming  level  perform¬ 
ance,  or  no  learning  effect.  The  Improvement  Curve  function  which  has 
remained  parochial  to  the  aerospace  industriee  has  been  presented  with 
the  eame  motive  as  the  rooster  who  showed  hie  hen  an  ostrich  egg--"It's 
not  that  I'm  complaining,  it's  just  that  I'd  like  you  to  see  what  others  are 
doing,'  " 
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THE  EFFECT  OF  RELIABILITY,  LENGTH,  AND  SCORE  CONVERSION 
ON  A  MEASURE  OF  PERSONNEL  ALLOCATION  EFFICIENCY 

Richard  C  ,  Sorenson  and  Cecil  D,  Johnson 
U.  S,  Army  Personnel  Re  search  Office 
Washington,  D,  C. 


Within  the  United  States  Army  it  has  been  realised  for  many  years  that 
an  effective  military  organization  must  have  the  right  kind  of  men  as  well 
as  the  most  advanced  and  effective  equipment.  Of  course  this  does  not  mean 
that  the  Army  must  have  only  the  'best'  of  the  personnel  pool,  but  does  mean 
that  those  men  taken  from  the  personnel  pool  must  be  mate  hed  with  jobs  in 
a  way  that  facilitates  maximum  manpower  utilization  There  are  two  sides 
to  this  task  of  manpower  utilization;  1)  the  various  functions  performed 
within  the  Army  must  be  analyzed  to  determine  the  different  skills  needed 
to  perform  those  functions,  and  2)  the  individual  differences  within  the 
personnel  pool  must  be  analyzed  to  find  those  different  abilities  that  can  be 
reliably  measured.  At  this  point  we  are  left  with  the  problem  of  developing 
effective  measuring  instruments  and  of  devising  ways  and  means  of  assign* 
ing  men  to  job6  on  the  basis  of  the  measure  of  abilities.  This  whole  attack 
on  manpower  utilization  rests  on  the  realization  that  while  few  men  can  be 
trained- -no  matter  how  extensive  and  careful  the  training- -to  do  all  the 
Army  jobs  as  well  as  those  who  do  them  best,  most  men  accepted  by  the 
Army  can  be  trained  such  that  they  are  effective  in  performing  those  skills 
for  which  they  are  most  apt,  and  when  properly  assigned,  will  be  an  asset 
to  the  Army. 

Thus  the  solution  of  the  problem  rests  on  successfully  accomplishing  the 
following;  1)  identifying  job  families  within  the  Army  that  require  personnel 
with  different  ability,  2)  identifying  and  measuring  these  abilities  within 
the  personnel  pool,  3)  estimating  the  performance  on  the  job  on  the  basis 
of  measures  of  ability  related  to  job  requirements  and  4)  assigning  men 
to  jobs  so  as  to  maximize  overall  performance, 

The  first  of  these  steps  has  been  treated  in  the  establishment  of  the 
Army  occupational  areas.  Ten  occupational  areas  have  been  identified 
and  shown  to  be  satisfactory  in  classifying  the  various  Army  functions 
assigned  to  enlisted  men  (EM)  [10]  # .  Recent  research  indicates  that  nine 


*The  numerals  in  brackets  indicate  numbered  references  listed  at  end  of 

paper. 
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categories  of  training  schools  within  Army  Advanced  Individual  training 
may  be  differentiated  [5]  ,  It  may  be  assumed  that  continuing  research  will 
be  required  to  evaluate  the  constantly  changing  functions  performed  by 
Army  EM  as  new  methods  and  procedures  are  introduced. 

The  Army  Classification  Battery  (ACB)  has  been  developed  to  measure 
aptitudes  related  to  Army  jobs  [4]  .  An  important  research  mission  of 
USAPRO  is  to  introduce  new  measuring  devices,  and  to  revise  and/or 
validate  present  tests  [7]  , 

The  eight  current  Aptitude  Areas  are  functions  of  the  eleven  tests 
within  the  ACB  and  serve  as  performance  estimates  for  the  Military 
Occupational  Specialties  (MOS)  in  one  or  two  occupational  categories, 

These  Aptitude  Area  Scores  are  currently  used  for  differential  classifica¬ 
tion  [10]  .  (See  Figure  1.  ) 

The  benefits  inherent  in  differential  classification  using  Aptitude  Area 
Scores  stem  from  the  fact  that  information  is  obtained  relative  to  the 
differences  in  ability  between  individuals  and  to  differences  within  the 
individual.  Thus  EM  may  be  assigned  to  jobs  for  which  their  probability 
of  success  may  be  a  good  deal  greater  than  that  for  Army  jobs  in  general. 

The  technical  gain  is  twofold.  First,  a  given  level  of  aptitude  for  a 
given  job  can  be  assured  by  a  lower  score  on  the  specific  selector  highly 
related  to  the  Job  than  would  be  required  to  maintain  the  same  standard 
of  excellence  if  the  selection  were  based  on  an  instrument  less  valid  for 
the  purpose  at  hand,  Secondly,  when  recruits  are  taken  above  a  given 
cutting  score  on  a  general  selector,  they  are  removed  from  that  score 
interval  of  the  aptitude  pool  for  all  other  jobs  as  well,  However,  when 
recruits  are  taken  above  a  given  cutting  score  on  a  specific  selector, 
they  come  from  a  much  broader  range  of  scores  as  far  as  the  pool  for 
another  specific  selector  is  concerned.  To  the  extent  that  one  specific 
■  elector  is  uncorrelated  with  a  second,  the  entire  range  of  scores  is  still 
available  on  the  latter  after  selection  has  been  accomplished  on  the  first 
selector. 

Thus  we  see  that  for  a  particular  sample  of  1800  individuals  drawn 
for  the  purpose  of  standardizing  a  subsequent  version  of  one  of  the  tests 
56%  were  above  average  on  the  Armed  Forces  Qualification  Test  (AFQT) 
relative  to  the  original  standardization  population,  Of  this  same  sample, 
however,  91%  were  above  the  average  for  the  Aptitude  Area  in  which 
they  scored  highest,  (See  Figure  2.) 
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One  further  operational  gain  was  investigated.  Under  the  former 
system  in  which  a  single  test--the  Army  General  Classification  Test  (AGCT) 
--  was  practically  the  sole  determinant  of  Army  classification,  selection 
for  one  set  of  jobs  automatically  gave  those  jobs  the  upper  segment  of  the 
distribution  of  test  scores.  The  lower  segment  wac  left  for  the  remaining 
jobs,  In  the  operational  problem  filling  the  manpower  requirements  of  an 
infantry  division,  approximately  one  half  of  the  men  were  combat  infantry¬ 
men  .  If,  as  happened  at  times  during  the  war,  a  test  were  used  to  select 
primarily  for  the  noncombat  specialties,  these  jobs  would  be  filled  by  using 
the  upper  half  of  the  distribution  ..  In  such  a  case  only  the  lower  half  of 
the  distributions  of  test  scores  would  be  available  for  the  combat  jobs  as 
indicated  in  Figure  3.  However,  when  the  results  of  the  distribution  of 
men  into  aptitude  areas  corresponding  to  job  families  for  the  infantry  divi¬ 
sion  used  in  the  standardisation  study  mentioned  above  are  viewed,  the 
distribution  of  AGCT  scores  for  the  non-priority  or  combat  Jobs  is  seen 
to  be  almost  equal  to  the  distribution  of  AGCT  scores  for  the  priority  Jobe. 
This  is  shown  graphically  in  Figure  4, 

A  great  deal  of  research  has  been  undertaken  to  make  optimal  alloca* 
tion  feasible,  Various  versions  of  the  optimal  regions  and  other  methods 
are  now  available  for  operational  use  [lj  .  In  the  research  reported  in 
this  paper  a  routine  derived  from  the  Hungarian  solution  to  the  transporta¬ 
tion  problem  was  used  [8]  . 

In  this  paper  we  will  be  concerned  with  investigating  characteristics 
of  performance  estimates  (and  the  test  battery  from  which  they  were 
derived)  as  they  relate  to  the  criterion  of  personnel  allocation  efficiency 
as  measured  by  the  average  performance  under  conditions  of  optimal 
allocation,  This  measure  of  performance  is  the  objective  function  to  be 
maximized  in  the  transportation  problem,  Many  relationships  involving 
this  objective  function  and  the  variables  of  this  study  may  easily  be  calcu¬ 
lated  analytically  aaeumlng  ideal  conditions,  s.  g.  ,  continuous  normally 
distributed  psychological  test  scores.  For  instance  Brogden  [2,  3]  has 
shown  that  when  other  factors  are  held  constant  and  certain  conditions 
assumed,  the  efficiency  of  allocation  is  directly  proportional  to  the  validity 
of  the  performance  estimate,  and  that  one  may  determine  by  analytic  means 
the  allocation  efficiency  for  given  numbers  of  Jobs,  percent  of  personnel 
pool  rejected,  and  intercorrelation  of  performance  estimatee,  In  reality, 
however,  we  are  not  dealing  with  continuous  variables  and  frequently 
other  assumptions  are  not  met,  Also,  in  practice  the  scores  are  often 
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transformed  in  such  a  way  that  considerable  information  is  lost.  It  is  less 
easy  to  investigate  the  more  realistic  situations  analytically.  Thus  we  have 
embarked  on  a  program  to  study  by  a  Monte  Carlo  approach  the  general 
relationship  between  amount  of  information  in  a  distribution  of  discrete 
performance  estimates  and  the  performance  level  it  is  possible  to  achieve 
by  the  most  efficient  pattern  of  personnel  assignments. 

The  basic  step  in  the  implementation  of  a  statistical  experiment  is  the 
generation  of  uniformly  distributed  random  numbers.  We  have  used 
computer  routines  which  generate  pseudo-random  numbers  by  the  power 
residue  method  [9]  .  These  distributions  of  uniform  variables  are  then 
transformed  to  distributions  of  normal  variables.  This  transformation 
results  in  a  matrix,  X,  of  order  n  by  k,  i.e.  ,  n  entities  are  represented 
each  by  a  vector  of  k  simulated  scores: 


(1) 


X  = 


V  X12'  ' 
X21*  X22  ’ 


where 


(2) 


X'X  -  nl 

and  1  'X  -*  0  > 

when  n 


We  see  then  that  for  each  sample  we  generate  a  matrix  that  has  an 
expectation  for  its  covariance  matrix  of  the  identity  matrix. 


Now  we  desire  to  further  transform  the  matrix  X  by  post  multi¬ 
plication  by  a  matrix  T  such  that  the  resulting  matrix  has  for  its 
expected  covariance  matrix  a  given  matrix  C: 

\ 

XT  =  Y 

(3)  where  Y'Y  -  nC  l 


when 


n 
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The  matrix  C  is  specified  as  a  function  of  the  desired  standard  deviation 
and  intercorrelation  of  the  variables: 

(4)  C  =  s  R  s. 

Where  R  is  the  desired  correlation  matrix  and  s  is  the  diagonal  matrix 
of  standard  deviations , 

We  wish  to  find  the  matrix  T  such  that  the  conditions  in  (3)  will  hold. 
From  these  equations  we  may  write  the  requirement  that: 

(5)  (U  Y'Y  =  (j-)  T'X'XT  -  C 

n  n 


when  n  -*  » , 

From  (2)  we  see  that 

-X'X  -  I 

n 

(6) 

when  n  -*  * 

and  from  (5)  and  (6)  we  have 

(7)  T'T  =  C  , 

We  may  represent  the  matrix  C  in  terms  of  its  basic  structure: 

(8)  C  =  QAQ' 

where  QQ'  =  Q'Q  =  I. 

We  know  that  the  matrix  C  to  any  power  e.  g.  ,  i,  may  be  formed  by  raising 
the  eigen  values  of  C  to  that  power,  premultiplying  by  Q  and  post- 
multiplying  by  Q '  [  6  ]  : 


(9) 

cl  =  qa£g' 

(10) 

thus 

=  QA^Q' 
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Formula  (10)  could  be  demonstrated  as  follows: 

i  1  i.  1 

(11)  C2  C2  =  QA2  Q'  QA2  q'  =  QAQ'  =  C  . 


We  will  let 

_i 

(12)  T  =  C2 
We  see  that 

(13)  T'T  =  C2  =  C 

Hence  a  transformation  solved  for  by  equation  (11)  meets  the  requirement  s 
of  (7)  and  while  there  are  an  infinite  number  of  transformations  that  meet 
this  requirement  the  one  indicated  is  by  far  the  most  advantageous  since 
it  provides  for  uniformity  of  rounding  errors  and  impartially  improves 
normality  of  the  transformed  scores. 

Thus  we  may  simulate  samples  of  personnel  by  building  into  the  score 
distribution  characteristics  of  performance  estimates  in  which  we  are 
specifically  interested.  These  performance  estimates  may  in  turn  be  a 
function  of  such  test  characteristics  as  length,  reliability  and  validity. 

The  effectiveness  of  a  test  or  of  the  resulting  performance  estimation  is 
determined  by  its  potential  contribution  to  the  optimal  allocation  average, 
that  is,  the  average  estimated  performance  of  men  on  the  jobs  to  which 
they  are  assigned. 

Let  us  first  consider  one  of  these  characteristics  of  a  distribution 
of  performance  estimates;  namely  the  standard  deviation.  Often  times, 
in  the  course  of  personnel  operations  where  men  are  actually  being  assigned 
to  jobs  on  the  basis  of  measured  attributes,  distributions  of  scores  are 
transformed  from  distributions  in  which  there  are  two  or  three  significant 
digits  to  distributions  in  which  there  is  only  one  significant  digit.  This  is 
the  case  in  assigning  men  to  jobs  in  the  ^rmy,  The  three  digit  Army 
Aptitude  Area  Score  is  coded  according  to  AR  611-259  to  a  score  taking 
on  the  values  ranging  from  zero  to  nine.  The  questions  we  ask  are: 

1)  V.  ,at  loss  of  information  occurs  when  scores  are  coded  to  a  one  digit 
scale,  and  2)  What  affect  does  this  loss  of  information  have  on  average 
performance  when  these  scores  are  used  to  assign  men  to  jobs? 
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In  Figure  5  we  demonstrate  the  effect  of  coding  the  scores  of  a 
continuous  distribution  centered  at  50  into  nine  score  scales,  e.  g.  ,  entities 
with  scores  less  than  11.  5  were  given  a  coded  score  of  1,  entities  with 
scores  11.  5  or  greater  but  less  than  22.  5  were  given  a  coded  score  of 
2,  ...  entities  with  scores  88.  5  or  greater  when  given  a  score  of  9,  The 
upper  portion  is  the  resulting  distribution  when  the  original  distribution 
has  a  standard  deviation  of  20.  The  information  measure,  H,  has  an 
intuitive  appeal  because  it  is  sensitive  to  both  the  size  of  the  coded  interval 
and  the  spread  of  scores.  For  the  above  distribution  H  may  be  calculated 
by 


(14) 


9 

H  =  2  (p.  log  p.) 

‘  .1  1  1 


where  is  the  proportion  of  the  entities  in  the  ith  interval  and  log  p.,  is 

the  natural  logarithm  of  p^.  The  information  measure  corresponding  to 

the  distribution  represented  in  the  top  of  Figure  5  is  1,  991.  In  the  lower 
figure,  a  similar  transformation  was  performed  on  a  continuous  distribu¬ 
tion,  where  the  original  distribution  has  a  standard  deviation  of  10.  We 
see  here  that  the  cases  are  primarily  distributed  in  intervals  4,  5,  and  6, 
that  they  are  much  more  closely  grouped  together.  That  much  more 
information  is  lost  is  indicated  by  the  corresponding  information  measures 
which  is  1.  372.  We  may  note  that  the  maximum  value  for  the  information 
measure  corresponding  to  a  nine  score  scale  is  2.197  which  occurs  when 
the  distribution  is  uniform. 


Now  we  can  easily  see  that  information  is  lost  when  we  go  from  several 
significant  digits  to  one  significant  digit.  We  also  see  that  more  informa¬ 
tion  is  lost  when  the  standard  deviation  of  the  parent  distribution  is  small 
than  when  it  is  large.  We  desire  to  investigate  the  degree  to  which  such 
information  loss  affects  the  optimal  allocation  average. 


Another  variable  of  interest  is  the  quota  restriction  places  on  the 
optimal  allocation.  A  natural  quota  is  defined  as  the  number  of  men  that 
would  be  assigned  to  a  job  if  everyone  were  assigned  so  as  to  maximize 
his  individual  performance  without  regard  to  quotas.  In  the  case  of  equal 
variances  and  intercorrelations  among  performance  estimates,  the  natural 
quotas  are  equal,  i.  e.  ,  uniform.  On  theoretical  grounds  we  can  conclude 
that  the  degree  to  which  the  quotas  are  perturbed  from  the  natural  is 
related  to  the  allocation  average.  However,  the  effect  of  this  quota  factor 
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on  the  other  relationships  must  be  studied  empirically.  We  see  in  Figure  6 
the  percentage  quotas  imposed  on  optimal  allocation  for  the  situation  where 
we  have  16  jobs  and  where  we  simulate  only  4  jobs.  Note  that  the  natural 
or  uniform  quota  for  16  jobs  is  .  0625.  That  is  the  proportion  of  the  total 
personnel  pool  that  would  be  allocated  to  each  job.  For  4  variables  it  is 
.25.  There  are  two  considerations  that  determined  the  perturbed  quotas. 
The  first  was  that  we  wanted  at  least  one  individual  to  be  assigned  to  each 
job,  for  both  the  16  and  4  variables  for  each  of  the  sizes  of  samples.  The 
second  was  that  we  wanted  the  ratio  of  the  information  measure  that  was 
found  to  exist  between  the  4  and  16  variable  situation,  for  the  natural  quotas, 
to  exist  also  for  the  perturbed  quotas.  We  required  that  the  uncertainty 
of  assigning  men  to  jobs  with  16  variables  be  twice  that  for  assignment 
with  4  variables  for  both  the  natural  and  perturbed  quotas.  The  resulting 
proportions  indicated  in  the  table  were  the  result  of  the  two  considerations 
mentioned  above.  We  feel  that  in  imposing  these  quota  restrictions  in 
our  experiment  we  are  being  realistic,  in  that  the  necessary  perturbations 
in  the  quotas  in  the  actual  operational  conduct  of  the  Army  personnel  system 
would  not  be  greater  than  this. 

5 

In  order  to  study  these  effects,  a  2  factorial  experiment  using 
simulated  performance  estimates  was  designed.  The  five  factors  were: 

(l)  standard  deviation  of  the  estimated  performance;  (2)  number  of  cases 
in  the  sample;  (3)  number  of  variables;  (4)  number  of  score  intervals; 
and  (5)  quota  restriction.  Figure  7  indicates  the  various  levels  of  the 
five  factors  that  were  used.  The  performance  estimate  variables  were 
generated  such  that  they  had  an  expectation  of  .  70  for  their  intercorrelation. 
For  those  samples  that  were  randomly  assigned  to  Level  a  of  Factor  1, 
the  parent  distribution  was  generated  to  have  a  standard  deviation  of  10; 
for  those  assigned  to  Level  b,  the  standard  deviation  was  20.  Similarly, 
those  samples  assigned  to  the  first  level  of  Factor  4  were  transformed 
to  have  9  score  intervals,  while  those  assigned  to  Level  b  were  transformed 
to  have  99  score  intervals.  The  number  of  cases  and  variables  represented 
correspond  to  the  level  of  Factors  2  and  3  to  which  the  sample  was  assigned. 
Those  samples  assigned  to  Level  a  of  Factor  5  were  allocated  with  uniform 
quotas.  Those  samples  assigned  to  Level  b  were  allocated  with  perturoed 
quotas.  Thus  we  have  a  2^  factorial  experiment  in  which  there  are  32 
cells.  The  experiment  was  initially  replicated  10  times.  Three  hundred 
and  twenty  samples  were  generated  from  a  simulated  personnel  pool  and 
allocated  optimally  to  either  4  or  16  job  categories.  Figure  8  is  a  flow 
diagram  indicating  the  five  steps  in  this  experiment.  In  step  1,  the  matrix 
X  of  normally  distributed  random  numbers,  was  generated.  In  the  second 
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step,  the  matrix  Y  of  continuous  performance  estimates,  was  derived 
by  multiplying  the  matrix  X  by  the  transformation  matrix.  The  continuous 
performance  estimates  were  used  in  evaluating  the  allocations  under  the 
various  experimental  conditions  by  averaging  the  estimated  performance 
of  men  on  the  jobs  to  which  they  were  assigned,  In  doing  this,  we  used 
the  continuous  performance  estimates,  since  continuous  performance 
estimates  yield  an  unbiased  estimate  of  the  actual  performance  of  men  on 
the  job,  whereas  discrete  performance  estimates  would  have  introduced  a 
slight  bias.  As  may  be  seen  from  the  arrow  going  from  step  2  to  step  5 
in  the  graphical  presentation,  the  continuous  performance  estimates  were 
used  in  the  calculation  of  the  allocation  average.  Instep  3,  the  matrix  7 
was  derived  by  forming  a  discrete  performance  estimate  from  the 
continuous  performance  estimate.  This  was  done  simply  by  forming  the 
scores  into  either  9  or  99  score  intervals,  Step  4,  the  allocation  step, 
was  accomplished  by  a  computer  program  which  optimally  allocates  men 
to  jobs  by  a  linear  program  derived  from  the  Hungarian  Solution  to  the 
transportation  problem  [8]  .  The  average  performance  for  men  who  are 
thus  allocated  is  then  calculated,  It  is  these  allocation  averages  which 
are  subjected  to  the  analysis  of  variance  in  this  experiment. 

We  have  put  the  analysis  of  variance  to  a  slightly  different  use  in 
our  experiment  than  is  the  usual  case,  Theoretical  considerations  in 
this  experiment  dictate  that  we  should  expect  significant  differences 
between  the  two  levels  of  each  of  these  five  factors,  We  are  not  testing 
to  see  if  the  null  hypothesis  should  be  rejected,  but  we  are  performing 
the  analysis  of  variance  so  that  in  the  event  that  the  main  effects  are 
not  significant,  we  can  evaluate  our  simulation  for  its  adequacy  with 
regard  to  the  number  of  replications,  Thus,  the  purpose  of  the  analysis 
of  variance  in  this  experiment  ie  primarily  that  of  evaluating  the  number 
of  replications  that  we  used  in  our  simulation.  With  10  replications,  four 
of  the  five  factors  were  highly  significant  at  the  ,  001  level  or  less, 
However,  the  effect  of  Factor  2,  the  number  of  cases  in  each  simulated 
personnel  sample,  was  not  significant.  We  then  repeated  the  experiment 
using  as  the  level  of  Factor  2  different  siies  of  aamples;  32  and  192, 

We  found  that  while  there  was  a  small  difference,  this  difference  was 
insignificant  both  statistically  and  practically.  We  conclude  that  when 
allocating  large  quantities  of  men  to  jobs  under  the  condition!  specified 
above,  we  are  justified  in  aub-optimiaing  (random  sampling  the  overall 
sample  into  several  eubsamples  and  allocating  each  of  the  eubsamples 
optimally).  In  ao  doing,  we  may  operate  with  lees  computer  space  with 
little  concern  for  the  loss  in  allocation  average, 
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In  Figure  9  we  have  shown  the  mean  performance  for  the  levels  of 
those  factors  that  were  found  to  be  statistically  significant.  The.  results 
indicated  that  the  number  of  variables  is  the  most  important  of  the  factors 
of  the  experiment.  We  could  increase  the  gain  over  random  allocation 
by  72%  by  increasing  the  number  of  criterion  variables  from  4  to  16.  This 
indicates  that  one  of  the  most  promising  avenues  of  psychometric  and 
personnel  research  is  to  differentially  predict  more  job  categories  or  job 
families  than  we  are  now  doing.  The  number  of  score  intervals  factor 
was  a  significant  one  as  was  the  quota  factor.  However,  the  latter  was 
of  no  practical  significance.  We  feel  that  we  may  continue  to  use  natural 
(or  uniform)  quotas  in  our  research  work  and  generalize  our  interpretation 
of  results  to  realistic  situations  where  the  quotas  are  not  uniform. 

The  interactions  of  Factor  1  with  Factor  4,  and  Factor  3  with  Factor  4 
were  both  significant  at  the  .  01  level.  The  cell  means  for  these  two 
interactions  are  found  in  Figures  10  and  11.  It  appears  that  the  information 
loss  is  considerably  more  crucial  when  we  are  dealing  with  16  differential 
job  predictions  than  when  we  are  dealing  with  only  4.  The  significant 
interaction  between  Factor  1  and  Factor  4  indicates  that  the  loss  in  the 
allocation  average  going  from  99  score  intervals  to  9  score  intervals  is 
much  greater  when  the  standard  deviation  is  10  than  when  it  is  20.  (Recall 
that  this  was  predicted  from  considerations  of  the  amount  of  information 
in  the  respective  distributions.)  The  results  thus  far  indicate  that: 

(l)  mean  performance  may  be  increased  by  increasing  the  number  of 
differential  performance  estimates,  (2)  when  attempting  to  do  1,  it  is 
important  that  all  the  information  possible  be  retained  in  the  score 
distribution  by  using  as.  many  score  intervals  as  is  meaningful,  and 
(3)  in  going  from  a  99  interval  distribution  to  a  9  interval  one,  the  loss 
is  doubled  if  the  original  standard  deviation  is  10  rather  than  20. 

These  results  may  be  evaluated  from  at  least  two  points  of  view; 
first,  from  that  of  an  agency  dealing  with  actual  score  distributions,  and 
second,  from  the  point  of  view  of  the  test  constructor.  He  looks  at  our 
number  of  intervals  factor  as  the  number  of  items  in  a  test,  since  the 
number  of  meaningful  score  intervals  is  related  to  the  number  of  test 
items.  Furthermore,  he  may  consider  our  standard  deviation  factor  in 
terms  of  the  relationship  between  the  standard  deviation  of  a  test  and  the 
reliability  and  number  of  items  in  the  test. 
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Upon  consideration  of  the  factors  mentioned  above,  an  additional 
experiment  was  designed.  The  factors  to  be  studied  and  their  levels  are 
indicated  in  Figure  12.  Ten  samples  of  200  entities  were  assigned  to  each 
of  the  eight  cells  of  the  design  formed  by  the  first  three  factors.  Each 
sample  was  optimally  allocated  and  evaluated  at  each  level  of  Factor  4. 

For  each  sample,  vectors  of  test  scores  were  generated  and  transformed 
to  represent  perfectly  valid  performance  estimates. 

Figure  13  represents  by  a  flow  diagram  the  steps  followed  in  the 
experiment.  First,  the  matrix  of  normal  random  numbers ,  X,  was 
generated.  In  step  2,  X  was  transformed  to  a  matrix  of  continuous  test 
variables.  In  step  3  the  continuous  test  variables  were  formed  which 
were  to  be  used  in  the  evaluation  of  our  allocations  in  step  8.  In  step  4, 

/v 

the  discrete  test  variables,  G,  were  formed  from  the  continuous  test 
variables,  matrix  G,  by  creating  either  20  or  40  discrete  score  intervals. 
From  G,  the  performance  estimates,  Y,  were  formed  by  the  appropriate 
regression  equation.  These  performance  estimates  were  used  in  allocating 
the  men  to  jobs  in  step  7.  In  step  6,  the  performance  estimates  were 
transformed  to  stanine  form  and  again  the  men  were  allocated  to  jobs  and 
the  allocation  was  evaluated. 

Note  that  this  analysis  of  variance  is  a  split  plot  analysis  of  variance 
in  which  we  can  analyze  the  between- samples  variance  and  the  within  - 
samples  variance.  First,  let  us  look  at  Figure  14,  which  reports  the 
results  of  the  between -samples  variance.  The  effect  of  intercorrelations, 
reliability,  and  the  inter-action  between  intercorrelations  and  reliability, 
were  all  significant.  The  number  of  items  was  significant  only  at  the  .  25 
level,  with  10  replications.  We  see  from  the  analysis  of  the  within- samples 
variance  (see  Figure  15)  that  the  score  conversion  factor  was  significant 
and  the  score  conversion-reliability  interaction  was  significant  as  were 
the  three  factor  interactions  of  score  conversion,  intercorrelation, 
reliability  and  score  conversion,  reliability:,  number  of  items.  Let  us 
now  look  at  the  difference  in  the  mean  job  performance  for  the  two  levels 
of  each  of  the  four  factors  as  indicated  in  Figure  16.  It  is  of  interest  to 
note  that  by  reducing  the  intercorrelation  among  the  test  variables,  a 
great  increase  can  be  brought  about  in  the  allocation  average  (i.  e.  ,  mean 
job  performance).  We  see  also,  that  the  test  reliability  is  an  important 
consideration.  Let  us  note  that  the  difference  in  mean  performance  for 
the  two  different  levels  of  number  of  items,  apart  from  validity,  inter¬ 
correlation,  and  reliability,  was  in  the  direction  that  the  larger  the  number 
of  items,  the  higher  the  allocation  average.  The  difference  across  the  two 
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levels  of  score  conversion  (i.  e.  ,  no  conversion  vs.  a  conversion  from  the 
score  to  the  stanine)  was  also  a  significant  one.  As  we  look  at  the  inter¬ 
action  between  the  intercorrelation  among  the  test  variables  and  the 
reliability  (see  Figure  17),  we  see  that  the  reliability  is  a  more  crucial 
consideration  when  high  intercorrelations  prevail  than  when  they  are  low. 

Inasmuch  as  we  did  not  find  the  number  of  items  to  be  a  significant 
consideration,  we  replicated  the  experiment  for  crucial  cells  20  more  times. 
In  Figure  18,  we  see  the  results  of  that  analysis  of  variance.  We  see  that 
the  number  of  items  is  significant,  and  that  the  score  conversion  as  well 
is  statistically  significant.  In  looking  at  the  means  for  that  experiment, 
we  find  that  as  we  go  from  40  items  to  20  items,  that  is,  when  we  cut  the 
length  of  the  test  in  half,  even  if  we  would  keep  the  reliability  of  the  test 
the  same  and  the  validity  of  the  test  the  same,  we  would  lose  approximately 
8%  of  our  gain  over  random  allocation  of  men  to  jobs, 

The  results  of  this  work  indicate  that  the  use  of  caution  is  warranted 
in  advocating  the  use  of  shorter  tests  in  optimal  differential  classification, 
even  if  the  shorter  tests  retain  the  reliability  and  validity  of  the  longer 
tests,  especially  if  the  reliability  of  the  tests  is  closer  to  .  7  than  to  .  9. 

This  and  other  research  currently  in  progress  has  impact  on  the  planning 
of  further  test  development  research  and  on  the  operational  handling  of 
test  scores  and  performance  estimates.  Furthermore,  it  demonstrates 
that  simulated  experiments  can  yield  information  concerning  possible 
trade-off  between  allocation  average,  testing  costs,  and  the  relative  costs 
of  test  development.  Even  more  efficient  experiments  could  be  done  to 
estimate  the  magnitude  of  differences  by  employing  variance  reduction 
methods.  One,  the  regeneration  of  the  same  sample  transformed  for 
each  cell  in  the  design,  would  be  especially  appropriate  for  this  type  of 
study.  It  was  not  used  in  this  project  because  the  model  for  analysis  of 
variance  does  not  provide  for  a  residual  estimate  of  variance.  Future 
projects  will  employ  variance  reduction  techniques. 
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ARMY  CLASSIFICATION  BATTERY  I  ARMY  APTITUDE  AREAS 


Figure  I.  Any  Classification  Battery  (ACB)  tests  and  Any  Aptitude  Areas  as  function* 
of  ACB  variables. 
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50th  percentile  (a)  on  AFQT  and,  (b)  on  their  highest  aptitude  area 
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Figure  4.  Distribution  of  Army  Standard  Standard  Scores  on  overall  general  ability  when 
assignment  is  based  on  battery  of  tests* 
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Fl|u»  3.  Dlacrata  dlatrlbuclooa  raaultlni  froa  continucui  dlatrlbutlona  with 
aeandard  deviation  sf  10  and  at  20. 
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Figure  7.  Experimental  conditions  used  in  the  five-factor  experiment. 
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Figure  8.  Flow  diagram  of  experiment  using  simulated  performance  estimates 


Results  of  Experiment  Using  Simulated 
Performance  Estimates 


210 


m 


o  h- 

rH  iH 
rH  rH 


cm  tn 


13 

© 

-m  : 
c/j 


in  o 
o  r- 


0(0 

in  oo 


CM 

to 


r-  co 
(0  (0 


o  o 

iH  CM 


10 


0)  0) 
0> 


CO  CO 


« 

4a  h 
L  © 
3  L. 
■H  3 
U  +» 
©  © 
0b  x 


© 

> 


© 

> 


c 

c 

L 

L 

o 

e 

ft 

© 

IH 

ttm 

N 

© 

+• 

© 

© 

e 

c 

© 

© 

H 

H 

IH 

in 

u 

■  ■■ 

IW 

41 

£ 

o 

> 

> 

© 

© 

0 

© 

■H 

© 

© 

•V 

U 

L 

u 

■3 

■3 

L 

L 

e 

o 

© 

© 

© 

u 

o 

u. 

•3 

•3 

> 

> 

© 

© 

L 

L 

© 

© 

4- 

4- 

4- 

4- 

*3 

*3 

0 

O 

e 

e 

© 

© 

C 

e 

4-> 

4-» 

© 

© 

• 

• 

• 

■ 

O 

O 

L 

U 

L 

L. 

3 

3 

CO 

CO 

as 

X 

X 

X 

O 

a 

} 


Standard  Deviation 


Number  of  Variables 


FACTORIAL  DESIGN  FOR  EXPERIMENT  USING  SIMULATED 
PSYCHOLOGICAL  TEST  VARIABLES 


c 

o 


© 

+■» 

© 

e 

k 

E 

k 

9 

♦ 

rH 

<n 

© 

M 

© 

O 

•  • 

•  mm 

• 

• 

+* 

o 

O 

k 

c 

o 

JZ 

•  mm 

CM 

©■ 

© 

© 

10 

k 

ii  ii 

© 

II 

n 

> 

k 

e 

© 

H- 

II 

II 

C 

o 

© 

H 

u 

v 

O 

o 

o 

4-1 

c 

k*  k* 

© 

lP 

c 

c 

o 

CO 

CO 

k 

k 

© 

© 

■H 

•  •  «• 

4-» 

•  • 

•  > 

A 

« § 

•  • 

k 

•  * 

•  • 

CO 

01  43 

© 

© 

43 

B 

© 

43 

o 

© 

JQ 

CD 

tt 

3 

o 

h- 

H  H 

H 

H 

H 

as 

H 

H 

V) 

H 

rH 

©  © 

© 

© 

© 

© 

© 

© 

>  > 

> 

> 

> 

> 

> 

> 

« • 

©  © 

•  • 

© 

© 

*  • 

© 

© 

•  • 

© 

© 

H 

-1  -J 

CM 

-J 

-J 

CO 

•J 

-J 

-J 

k 

k 

•  k 

k 

o 

© 

o 

p 

■H 

■k 

+* 

O 

O 

o 

u 

rt 

© 

© 

© 

u. 

u. 

u. 

Ll. 

Generate  |  X,  normal  random  numbers 


214 


0 

0 

O) 

0 

0 

CO 

+J 

© 

U 

© 

0 

a 

0 

H 

E 

c 

X 

> 

4-» 

0 

4-» 

0 

E 

i. 

w 

0 

0 

L 

+J 

C 

0 

u 

O 

O 

o 

■H 

0 

(0 

4- 

g 

tmm 

> 

0 

0 

L. 

+J 

CO  CO 

+1  0 

O 

©  0 

0 

3  © 

c 

0 

c 

a.  © 

C 

a 

O  iH 

o 

0  rH 

0 

4J 

© 

o 

3  JQ 

■H  JQ 

E 

©  0 

E 

rH 

c  a 

L. 

0  0 

u 

c  E 

C 

rH 

tmm  ■  mm 

© 

L.  — 

o 

O) 

0 

•H  u 

■H 

O  L, 

4- 

C  -H 

c  as 

• 

0  0 

L. 

0  0 

0 

0 

o  > 

U 

—  > 

0 

4-»  Q) 

0 

y 

O 

o 

■a 

a 

0 

0 

■H 

as 

m 

>- 

?«D 

l>-' 

<>-’ 

9k 

* 

E 

•  •  •  •  •  •  i  • 

hcvjco^  in  co  r-  co 


mdwloiicil  teat 


215 


ANALYSIS  OF  VARIANCE  OF  ALLOCATION  AVERAGE 


216 


Figure  15.  Analysis  of  variance  of  average  performances  for 
type  of  score  conversion  (within  sample  variance) 
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Figure  18.  ResuJ :s  of  analysis  of  variance  contrasting  number  of  items  at  a.  fixed 
level  of  reliability  and  intercorrelation  (.7  and  .4,  respectively). 
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QUANTITATIVE  ASSAY  FOR  CRUDE  ANTHRAX  TOXINS* 


Bertram  W.  Hrlnes,  Frederick  Klein,  and  Ralph  E.  Lincoln 
U.  R  Army  Rlnlosical  Laboratoriea 
Fort  Detrick,  Frederick,  Maryland 


ABSTRACT ,  The  whole  crude  toxin*  of  Baclllu*  anthracl*,  although 
apparently  reaponalble  for  the  death  of  animal*  with  anthrax,  had  never 
been  quantitated,  A  total  of  14  lota  of  the  toxic  culture  filtrate  of  B, 
anthracl*  were  pooled  into  one  large  lot  of  crude  anthrax  toxin*, 
extensive  a**ay  of  this  reference  material  wa*  conducted  in  four  labora¬ 
tories  by  use  of  the  time-to-death  of  the  intravenously  challenged  Flecher 
344  rat  a*  the  response  variable.  Doses  of  the  material  were  varied 
factorially  by  concentration,  dilution,  and  volume,  The  data  from  this 
study  were  used  to  define  a  potency  unit  of  the  crude  anthrax  toxins, 
Procedures  were  developed  and  illustrated  for  the  assay  of  unknown  lots 
of  the  toxins  by  comparing  the  rate  time-to-death  response  to  the  unknown 
with  either  (i)  the  responses  reported  in  this  study,  or  (ii)  directly  with 
the  rat  responses  to  a  new  sample  of  the  reference  toxins.  The  possibil¬ 
ities  and  limitations  of  this  standardisation  and  of  the  statistical  procedure 
through  which  it  was  developed  are  diicusasd. 

INTRODUCTION,  The  excellent  work  of  Smith,  Keppie,  and  Stanley 
(1955a),  demonstrating  the  toxins  of  Bacillus  anthracis  organisms  in  ths 
blood  from  guinea  pigs  in  the  terminal  siagea  ol  anthrax,  rekindled 
Interest  in  the  disease,  particularly  its  toxins,  (The  toxic  methabolic 
by-products  of  the  growth  of  B,  anthracis  are  compoeed  of  components  with 
different  biological  or  chemical  propertiee ,  Naturally  produced  combina¬ 
tions  of  these  components  in  unknown  proportions  will  be  referred  to  in 
this  paper  as  "toxins,  ")  To  date,  valid  comparisons  of  results  among  the 
several  experimenters  (Smith  st  al,  ,  1955a,  b,  1956;  Smith  and  Qallop, 
1956;  Thorne,  Molnar,  and  Strange,  I960;  Stanley  and  Smith,  1961;  Beall, 
Taylor,  and  Thorne,  1962;  Klein  et  al,  ,  1962;  Keppie,  Smith,  and  Harris- 
Smith,  1955;  Eckert  and  Bonventre,  1963;  Harris-Smith,  Smith,  and 
Keppie,  1958;  Sergeant,  Stanley,  and  Smith,  I960;  Stanley,  Sergeant,  and 
Smith,  I960)  who  have  reported  work  with  the  toxic  materials  produced  by 
B,  anthracis  have  been  difficult,  because  either  whole  crude  toxins  or  ths 
several  components  have  been  assayed  by  different  methods,  in  different 
assay  animals,  and  with  no  reference  standard  of  the  toxins, 


*This  paper  appeared  previously  in  the  Journal  of  Bacteriology,  Volume 
8 9 1  No,  1,  pages  74-83,  Permission  of  ths  edUors  of  this  journal  to 
publish  this  peper  in  these  Proceedings  is  appreciated. 
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Thia  paper  preaenta  the  reaulta  of  atudiea  to  quantitate,  in  term*  of 
defined  potency  unit*,  the  lethality  of  anthrax  toxin*  in  Fiacher  344  r*?;. 
The  author*  developed  a  reference  tot  of  atabiliaed  freeae-dried  crude 
anthrax  toxina,  Thia  reference  material  waa  uaed  in  th*  study  described 
here,  and  ia  available  for  other  atudiea  againat  which  samplei.  of  anthrax 
toxina  of  unknown  concentration  can  be  aaeayed, 

MATERIALS  AND  METHODS,  Animal ■>  Fiacher  344  albino  rata 
weighing  200  to  300  g  were  obtained Tx am ' the  Fort  Detrick  colonies  of 
Frank  Baall  and  Frederick  Klein.  Both  colonies  art  maintained  through 
brother-sister  matings  descended  from  the  colony  described  by  Taylor, 
Kennedy,  and  Blundell  (1961),  Thia  weight  range  waa  chosen,  because 
preliminary  data  indicated  that  the  reaponae  time  of  rata  that  weigh  more 
than  300  g  was  significantly  greater  than  that  of  rats  weighing  more  than 
200,  butleaa  than  300,  g,  Further  study  on  rata,  carefully  selected  for 
weight,  revealed  no  significant  diffaranc*  within  tha  weight  range  of  200 
to  300  g  (Table  1).  The  analysis  of  varianct  ia  presented  in  Tabl*  2. 

TABLE  1 

Response  time  in  minutes  of  2?  rata  injected  with 
1  ml  of  crude  anthrax  toxins  by  weight 
of  rat, 


"w \cAm 

!  of  rat 

260 

m 

S0C 

99 

102 

100 

97 

81 

94 

96 

80 

88 

94 

79 

105 

93 

78 

90 

92 

114 

101 

89 

76 

78 

88 

102 

82 

87 

71 

86 

833» 

783 

824 

92, 

84,  9 

90,  7 

*  Totals 

**  Harmonic  means , 
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TABLE  2 

Analyoio  of  variance  of  recipror*!  response  times 
recorded  in  Table  1 


Source 

df" 

Sum  of 
■quares 

Mean 

equate 

T 

Between  weighta  .... 

2 

,  0485 

.  0242 

1,  50** 

Within  weighte . 

24 

.  3859 

,  0161 

Total . 

26 

.4344 

— 

*  Degress  of  freedom. 
**  Not  significant. 


Rat  lethal  test.  Toxins  of  B_,  anthracia  were  injected  into  the  dorsal 
vein  of  the  penis  of  the  FiecherTat.  In  describing  this  test,  Beall  at  al. 
(1962)  noted  a  definite  relationship  between  the  dose  of  the  toxins  injected 
and  time-to-death. 

Anti  serum.  Equine  hyperimmune  serum  (DH-1-6C)  prepared  by 
repeated  injections  of  spores  of  the  Sterne  strain  of  !B.  anthracis ,  was 
used  (Thorne  et  al,  ,  I960).  ' 

Preparation  of  anthrax  toxins.  The  medium  used  was  described  by 
Thorne  et  al ,  (1966),  and  was  made  with  triple 'distilled  water.  Subsequent 
to  his  original  description,  Thorne  (personal  communication)  has  suggested 
some  changes,  The  medium  used  in  this  study  was  as  follows, 

Nine  stock  solutions  (A,  B,  C,  D,  E,  F,  0,  H,  and  I)  were  prepared. 

All  stock  solutions  may  be  stored  at  4  C  for  indefinite  periods  of  time. 
Solution  A  contained  CaCl^'  2H20,  0.  368  g/500  ml  of  water;  B  contained 

MgSO^'THjO,  0, 493  g/500  ml  of  water;  C  contained  MnSO^' H^O ,  0.043 

g/300  ml  of  water;  D  contained  adenine  sulfate,  0,105  g,  and  uracil ,  0.070  g 
(both  solids  were  dissolved  in  100  ml  of  water,  and  the  total  volume  was 
made  up  to  500  ml), 

Solution  E  contained  thiamine  HC1,  0,  025  g/500  ml  of  water;  F  contained 
tryptophan,  2,600  g;  cystine,  0,600  g;  and  glycine,  0,750  g.  The  solide 
in  solution  F  were  dissolved  as  follows,  Tryptophan  was  dissolved  in  6  ml 
of  6  N  KC1,  Cystine  was  dissolved  in  100  ml  of  water.  Glydni  was  dis¬ 
solved  in  150  ml  of  water,  These  three  solutions  were  combined,  and  water 
was  added  to  bring  the  total  volume  up  to  500  ml, 
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Solution  G  contained  KH^PO^,  34,  0  g/500  ml  of  water;  v  contained 
K2IirC4<  43.  6  g/oOu  mi  of  water*)  1  contained  charcoal  (Norit  A), 

3.  75  g/500  ml  of  water, 

A  10-ml  amount  of  each  stock  solution,  except  that  containing  char¬ 
coal,  was  added  to  a  suitable  container;  and  3.  6  g  of  Casamino  Acids 
(Difco)  were  added.  The  volume  was  brought  up  to  1  liter  with  triple - 
distilled  water,  and  the  pH  of  the  medium  was  adjusted  to  6,  9  with 

1  N  H-SO^  or  1  N  NaOH  as  needed,  A  460>ml  amount  of  this  preparation 
was  dispensed  into  a  3-liter  Fernbach  flask;  2  ml  of  charcoal  suspension 
were  added,  and  the  preparation  was  autoclaved  for  20  min  at  15  psi, 

Inoculation  procedure.  A  5-ml  amount  of  20%  glucose  (sterilised 
by  filtration)  was  added  to  the  Fernbach  flask  containing  460  ml  of  steri¬ 
lised  basal  medium.  Each  flask  of  final  medium  was  inoculated  with 

2  X  10°  Sterne  strain  spores.  The  inoculated  flasks  wore  incubated 
statically  for  23  to  27  hr  at  37  C;  4  hr  after  inoculation  35  ml  of  9% 
NaHCOj  were  added  to  each  flask. 

This  final  culture  was  centrifuged  at  3,000  X  g  for  30  min.  The 
supernatant  fluid  was  decanted,  and  10%  horse  serum  was  added.  The 
solution  was  then  sterilised  by  filtration  through  an  ultrafine  glass  filter, 

A  preliminary  test,  to  determine  the  potency  of  each  of  14  toxic 
filtrates,  was  done  by  injecting  1-ml  samples  of  each  filtrate  intravenously 
into  two  rata,  The  response  (death)  times  of  the  rats  wara  considered  ae 
indications  of  ths  toxicity  of  each  batch.  Tho  total  volume  par  batch  and 
the  response  times  of  ths  test  rats  are  given  in  Table  3. 

The  14  toxic  filtrates  wars  combined,  and  a  sacond  preliminary  tsst 
was  conducted  on  the  pooled  material.  The  two  rats  used  in  this  teat 
died  in  104  and  117  min,  with  a  mean  response  time  of  110.  5  min,  Both 
raiponss  times  are  within  one  standard  deviation  of  ths  mean  of  all 
batches, 

The  pooled  toxins  were  dispensed  into  600  drying  ampoules  (40  ml), 
each  containing  10  ml  of  toxins.  Ampoules  were  shell-frosan  in  Dry  Ice 
and  alcohol  (-79  C),  Froaen  ampoules  were  placed  on  an  Amlnco  Dryer 
(American  Instrument  Co,  ,  Silver  Spring,  Md, ),  and  dried  under  vacuum 
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TABLE  3 


Volume  per  batch  and  response  time  of  rats 
challenged  with  toxins  t>y  oaten 


Batch 

Total  volume 

Response  time  (min) 

_ _ _ 

Rat  A 

' 

Rat  B 

Mean 

1 

97 

92 

94.  5 

2 

450 

107 

91 

99.0 

3 

450 

97 

96 

96.  5 

4 

460 

95 

_* 

95,  0 

5 

420 

122 

124 

123.  0 

6 

450 

114 

125 

119.  5 

7 

510 

116 

90 

103.0 

8 

410 

121 

120 

120.  5 

9 

370 

88 

82 

85,0 

10 

510 

90 

94 

92.0 

11 

465 

106 

94 

100.0 

12 

425 

106 

92 

99.0 

13 

425 

117 

121 

119.0 

14 

300 

100 

117 

108.5 

Total 

6,095 

. 

103.9** 

*  Missed  the  vein, 
**SD  ■  12,14, 


of  10  to  30  \x  of  mercury  for  IS  to  25  hr,  Ampoules  were  sealed  under 
vacuumi  packed  in  cardboard  containers,  and  stored  at  -20  C,  A  third 
preliminary  test  was  conducted  at  this  point.  One  randomly  selected 
ampoule  was  reconstituted  with  10  ml  of  triple -distilled  water.  A  1-ml 
amount  of  this  toxic  material  was  assayed  in  each  of  five  rats,  Their 
mean  response  time  was  117,  2  min,  To  further  test  the  toxicity,  0.  2  ml 
of  undiluted  and  of  serial  twofold  dilutions  of  the  reconstituted  material 
was  injected  lntradermally  into  the  shaven  sides  of  a  guinea  pig,  and 
observed  for  edematous  reaction,  The  material  reacted  at  a  dilution  of 
1*.  32,  and  can  be  expressed  according  to  Thorne  et  al,  (1960)  as  containing 
32  toxic  unite.  Additional  vials  were  reconstituted  to  4X  concentration, 
and  tested  on  immunodiffusion  plates  against  the  standard  spore  antiserum 
(Thorne  et  al.  ,  i960).  Three  individual  lines  of  precipitate  appeared  in 
parallel  arrangement  when  tested  with  a  linear  pattern.  The  strongest 
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precipitate  line  was  identified  as  the  protective  antigen  (factor  II)  compo- 
nent  when  compared  with  a  standard  (Beall  **  »!.  ,  1962),  An  undiluted 
sample  of  the  resuspended  material  had  a  protective  antigen  titer  of  1;  64 
against  the  standard  spore  antiserum. 

Reference  toxins.  These  preliminary  tests  constituted  quality  control 
measures  on  the  remaining  59?  vials  of  dried  toxic  filtrate.  As  a  result 
of  these  tests,  it  was  known  that  these  vials  contained  the  known  compo¬ 
nents  of  anthrax  toxins. 

Procedures,  The  toxins  were  assayed  independently  by  «a:h  of  four 
investigators.  The  procedures  followed  by  each  of  the  four  were  as 
similar  as  possible. 

The  characterization  of  the  dose-response  relationship  of  the  toxins 
in  Fischer  rats  was  based  on  an  assay  in  which  the  two  dose  factors  of 
amount  and  concentration  of  toxins  were  each  tested  at  several  levels  as 
follows:  (i)  five  levels  of  the  amount  of  toxins  designated  as  4  ml,  2  ml, 

1.  5  ml,  1  ml,  and  0.  5  ml;  (ii)  seven  levels  of  the  concentration  of  the 
toxins  designated  as  4X,  2X,  IX,  0.  5X,  0.25X,  0.125X,  and  0.0625X, 
where  IX  is  defined  as  the  concentration  resulting  when  1  ampoule  is 
reconstituted  to  10  ml  with  a  diluent  of  triple-distilled  water.  Dilution* 
beyond  IX  were  made  with  distilled  water  plus  10%  normal  horse  serum. 

The  7X5  factorial  combinations  of  the  several  levels  of  these  two 
factors,  plus  19  control  groups,  were  each  tested  in  two  Fischer  rats 
by  each  of  four  investigators  (Table  4).  Three  sets  of  control  animals 
are  not  shown  in  Table  4.  The  first  set  included  five  paire  of  rats.  Each 
pair  was  inoculated  with  one  of  the  five  amounts  of  diluent  along  (i.  e,  , 
triple-distilled  water  plus  10%  normal  horse  serum)  to  provide  assurance 
that  their  companion  animals  responded  to  toxins  as  opposed  to  the  inocu¬ 
lation  of  the  diluents.  The  second  set  Included  seven  pairs  of  animals. 
Each  pair  in  this  set  was  inoculated  with  1.  5  ml  of  one  of  the  seven  con¬ 
centrations  of  toxins  mixed  with  0,  5  ml  (l/3  by  volume)  of  specific 
antiserum  (Thorne  et  al.  ,  I960).  The  seven  pairs  of  animals  in  the  third 
set  of  controls  were  inoculated  with  1.  5  ml  of  one  of  the  seven  concentra¬ 
tions  of  toxins  mixed  with  0,  5  ml  of  normal  horse  serum.  These  animals 
provided  assurance  that  the  control  no,  2  animals  that  lived  were  saved 
by  the  antiserum  specific  against  anthrax  toxin*. 
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Each  investigator  required  32  ampoules  of  dried  toxins.  Each  of  the 
32  ampoules  was  opened,  and  reconstituted  with  2.  5  ml  of  diluent  nr»cft'?!ed 
to  4  C.  The  contents  of  all  32  ampoules  were  then  pooled,  providing  a 
total  of  80  ml  of  reconstituted  toxins  at  a  concentration  of  4X  (4  times  the 
original).  All  concentrations  of  toxins  were  maintained  continuously  at 
4  C.  To  make  the  next  dilution,  40  ml  of  the  pool  (4X)  were  combined 
with  40  ml  of  diluent  (triple -distilled  water),  This  provided  80  ml  of  toxins 
at  a  concentration  of  2X.  Further  serial  twofold  dilutions  were  made  to 
0.0625X  (1/16  X  original  concentration)  and  inoculated  as  planned. 

Each  investigator  required  108  rats.  These  rats  were  caged  in  54 
consecutively  numbered  cages,  each  containing  two  animals.  Each  of  the 
54  treatment  combinations  was  given  to  the  two  animals  in  one  cage  at  the 
same  time.  The  order  of  the  treatments  was  randomised  for  each  investi¬ 
gator,  Response  times-to-death,  in  minutes,  were  recorded  for  each  rat 
and  constituted  the  basic  data, 

RESULTS.  The  response  times  for  animals  are  presented  in  Table  4, 
Although  none  of  the  controls  appears  in  this  table,  none  of  either  the  first 
or  second  groups  of  control  animals  died,  Some  animals  in  the  third  con¬ 
trol  group  challenged  with  1,  5  ml  of  toxins  plus  normal  horse  serum 
responded  nearly  the  same  as  test  animals  challenged  with  1.  5  ml  of 
toxins,  The  mean  response  times,  in  minutes,  of  these  control  animals 
by  concentration  of  toxins  are  recorded  in  Table  5.  The  pattern  of 
responses  by  the  controls  provided  the  needed  assurance  that  the  response 
of  the  test  animals  was  specifically  to  the  toxins  of  B,  anthracis, 


TABLE  5 

Mean  response  time  by  doss  and 
concentrations  of  toxins 


Concn 
of  toxin 

Dose  (ml 

) 

Mean 

Control* 

4 

2 

1.  5 

1 

0,  5 

4X 

57.  5 

53,  5 

59.  0 

62,  3 

75,  0 

60,  7 

60,  0 

2X 

55.  2 

60,  7 

66,4 

75.  2 

105. 1 

69,  0 

70.  0 

IX 

61.  3 

74, 1 

85.1 

88,  0 

198,  7 

86,  3 

134.  0 

0,  5X 

74.  4 

121.  6 

136,  3 

24  7,  0 

s** 

151.  3 

154.  0 

Mean 

61,  3 

70.  3 

78,  3 

89,  4 

143,  5 

91,  3 

■^Control  was  1,  5  ml  of  toxins  plus  normal  horse  serum, 
,!I>!,A11  animals  survived, 


Design  of  Experiments 


229 


In  spite  of  carefully  controlled  procedures  and  techniques,  the  results 
from  one  laboratory  (technician  a)  wars  so  errsiic  that  they  wire  die 
regarded  in  any  further  analysis.  Inspection  of  these  date  showed  that 
technician  4  was  the  only  one  having  reversal  of  results;  i.e.  ,  a  greater 
amount  of  toxins  not  killing  and  lesser  amounts  killing,  or  only  one  of  the 
two  tost  animals  responding  (except  at  doses  eliciting  a  response  above 
300  min).  These  extremely  variable  result!  indicated  that  adequate  con* 
trols  on  technique  and  environment  were  not  maintained  in  this  laboratory. 

The  reciprocals  of  the  response  times  were  used  for  analysis,  because 
reciprocal  response  times  are  nearly  normally  distributed  with  equal  vari- 
ances,  whereas  the  untransformed  response  times  are  positively  skewed 
with  unequal  variances  (Finney,  1952),  The  analysis  of  variance. pn  the. 
reciprocal  response  times  of  120  rats  from  the  four  highest  concentrations 
and  ths  fivs  doses  is  shown  in  Table  6,  From  this  analysis  it  was  easn 
that  both  doss  lsvel  and  eoncantration  had  atatiltically  significant  sffsets 
on  ths  response  time  of  Fischer  rats  injected  intravenously  with  anthrax 
toxins,  ,  . 


TABLE  6 

Analysis  of  variance  of  reciprocal  response  times  ^ 


Line 

no. 

Effect 

df 

Sum  of 

squares 

Mean 

square 

F* 

1 

Dose  (D) 

4 

11.9272 

2,9818 

229.  37-  * 

2 

Concentration  (C) 

3 

16. 5629 

5.  5210 

4  24.  69** 

3 

Technician  (T) 

2 

0.1  543 

0.0772 

5. 94*** 

4 

D  X  C 

12 

1.7984 

0,1499 

11.  53**  ' 

5 

D  X  T 

8 

0.1485 

0.  0186 

1.  43 

6 

D  X  T 

6 

0.  1180 

0.  0197 

1.  52 

7 

24 

0,  6452 

0. 0269 

2.  07 

8 

Error 

60 

0.  7814 

0,0130 

9 

Total 

■  .—  _  .,  -■  .. 

119 

_  _ _ j 

32.1  360 

*  Error  line  8  was  used  to  test  all  effects. 

**  Approximate  probabilities  <  0.001. 

***  Approximate  probabilities  <  0,  05. 
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The  analysis  further  showed  an  interaction  b«tw««*n  *>r.d  csr.ccr.tri 

non  to  be  itatlatically  significant.  The  mean  response  times  by  does  and 
concentration  of  toxins  are  given  in  Table  5,  From  the  tabled  means,  it 
can  be  seen  that  the  magnitude  of  this  interaction  is  slight  and  had  no 
practical  significance  in  the  further  analysis  and  interpretation  of  these 
data. 

The  analysis  also  showed  a  statistically  significant  difference  among 
technicians.  Inspection  of  the  data  showed  that  mean  reeponse  times  for 
all  rats  responding  for  technicians  1,  2,  and  3  were,  respectively,  78, 

83,  and  83  min,  Thie  ie  a  practically  unimportant  difference  which  we 
believe  may  in  part  be  due  to  environmental  factors,  because  genetic  dif¬ 
ferences  would  be  almost  nil  after  100  generations  of  Inbreeding.  The  rata 
used  by  technician  1  came  from  the  Beall  colony,  which  was  maintained 
in  a  different  environment  than  the  Klein  colony  animals  used  by  ths  othsr 
two  technicians.  Thie  raised  the  queation  as  to  the  effect  on  this  assay  of 
Fischer  rats  procured  from  non-Detrick  sources,  To  examine  this  effect , 
commercially  available  Fischer  rats  obtained  from  two  breeders  were 
tested  and  found  to  be  suitable  for  thie  asaay.  In  this  study,  20  Fischer 
344  rats  from  each  of  two  suppliers  (Microbiological  Associates,  Inc,  , 
Bethaada,  Md. ;  and  Charles  River  Breeding  Laboratories,  Inc.  ,  Brook¬ 
line.  Mass. )  were  challenged  in  each  of  two  laboratories,  The  response 
times  of  all  80  rats  axe  reported  in  Table  7,  No  statistically  significant 
difference  in  times  of  response  for  animals  from  the  two  suppliers  was 
observed,  A  difference  between  the  two  operators  and  ths  interaction  of 
operator  X  supplier  was  statistically  significant  at  ths  5%  level.  The 
mean  response  time  of  three  of  the  four  groups  differed  by  less  than  1  min, 
and  ths  fourth  group  differed  by  approximately  5  min,  This  difference  of 
about  5  min  between  thee#  two  groups  could  be  caused  by  a  difference  of 
about  seven  unite  of  toxins,  which  is  well  within  the  98%  confidence  limits 
of  an  estimated  potency.  Thue,  this  difference,  although  statistically 
significant,  was  considered  of  no  consequence  concerning  this  assay, 

A  test  to  determine  the  storage  characteristics  of  the  reference 
toxins  was  conducted  on  a  vial  of  the  toxins  which  had  been  stored  for 
36  months,  The  test  vial  was  reconstituted  with  10  ml  of  triple-dietilled 
water.  Six  rats  were  then  challenged  with  these  reconstituted  toxins, 
according  to  the  protocol  described  in  this  paper. 
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TABLE  7 

Response  times  in  minutes  by  supplier,  operators, 

and  rats  ■ 


Charles  River 
Breeding  Labs.  , 

Inc. 

Microbiological 
Associates,  Inc, 

rvAIB 

1* 

2 

1 

2 

1 

83 

87 

91 

85 

2 

88 

84 

84 

89 

3 

86 

86 

91 

89 

4 

83 

82 

88 

83 

5 

91 

84 

89 

92 

6 

87 

89 

88 

84 

7 

94 

88 

90 

101 

8 

88 

83 

92 

87 

9 

87 

83 

96 

102 

10 

91 

86 

77 

87 

11 

105 

83 

89 

93 

12 

94 

85 

94 

79 

13 

92 

79 

90 

107 

14 

90 

81 

91 

88 

13 

98 

81 

91 

83 

16 

91 

85 

77 

90 

17 

82 

83 

97 

89 

18 

90 

87 

89 

88 

19 

83 

83 

82 

75 

20 

88 

83 

90 

86 

Harmonic 

mean 

response 

time 

89.28 

84.10 

SB.  30 

88.  42 

•  Operator  number, 
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The  estimate  of  potency\from  that  test  was  32.4  potency  units  per  ml 

•*4.  4.L  .  1  V  _  „  mm  «  a.4  M  'T1  *U  4  •  «»«««■  ••  f  A  19  «  «  V4  4  t  «t  «  «■ 

ml  set  up  in  the  definition.  Therefore,  it  was  concluded  that  the  reference 
toxins  had  not  changed  with  respect  to  potency  during  36  months  of  storage. 

Development  of  procedures  &r  direct  assay  method.  A  potency  assay 
should  be  based  on  dose  expressed  In  terms  of  well-defined  units,  No  such 
units  have  as  yet  been  defined  for  anthrax  toxins,  Varying  the  amount  of 
toxins  by  varying  either  dose  or  concentration  would  have  a  significant 
effect  on  the  response  time  of  rats;  however,  rate  injected  with  1  ml  of 
toxins  concentrated  to  2X  responded  in  about  the  same  time  (75  min)  as 
rats  injected  with  2  ml  of  toxins  concentrated  at  IX  (74  min).  This  rela¬ 
tionship  holds  true  for  most  other  dose -by-concentration  combinations  for 
which  the  product  of  these  two  factors  is  a  constant.  If  doses  are  converted 
into  0.  5-ml  units,  and  concentrations  into  0,  0625  units,  then  the  doses 
and  concentrations  in  Table  4  can  be  expressed  as  shown  in  Table  8. 


TABLE  8 

Derivation  of  potency  units  of  anthrax  toxins 


Concn  of  toxins  in 

0.  0625-fold  units 

Dose  of  toxins  in  0.  5- 

ml  units 

8 

4 

3 

1 

64 

512 

64 

32 

236 

32 

16 

128 

16 

B 

64 

32 

24 

8 

4 

32 

16 

12 

4 

2 

16 

6 

2 

1 

8 

H 

3 

2 

1 

The  products  of  the  marginal  numbers  in  Table  8  for  any  two  equiva¬ 
lent  dose -by-concentration  combinations  are  the  same;  thus,  the  product 
of  two  dose  units  and  32  concentration  units  gives  64  total  potency  units 
of  toxins,  Similarly,  four  dose  un<ts  of  16  concentration  units  also  con¬ 
tain  64  total  potency  units  of  toxins.  We  define  the  potency  unit  of 
anthrax  toxins  to  be  expressed  as  these  products  of  dose  by  concentration 
of  this  particular  lot  of  toxins, 
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*  If  \vn  were  to  carry  the  definition  of  a  potency  '\nit  no  further,  then 
1  mi  oi  i*  concentration  of  any  enihra*.  tuxir*;,  rcgirdlcsc  :£  its  sctu"! 
effect  in  animals,  would  have  32  potency  units.  To  standardise  a  potency 
unit,  it  is  necessary  to  describe  the  association  between  the  dose,  in  units, 
and  the  potency,  in  terms  of  a  biological  response  to  this  particular  lot 
of  anthrax  toxins,  The  potency  of  any  other  lot  of  toxins  may  then  be 
measured  by  comparing  the  response  to  a  known  .Vmount  of  the  test  toxins 
with  the  response  to  the  same  amount  of  the  reference  toxina. 

These  response  characteristics  were  described  as  the  dose-response 
relationship  when  measured  doses  of  these  toxins  were  injected  intrave¬ 
nously  into  Fischer  344  rats.  The  challenged  rsts  responded  by  dying 
at  a  time  that  is  shown  here  to  be  highly  dependent  on  the  dose  measured 
in  potency  units  of  these  toxins, 

The  regression  of  mean  reciprocal  response  times  on  the  log2  of 

the  potency  units  of  anthrax  toxins  is  shown  in  Figure  1.  The  least  squaras 
line  has  the  aquation; 

(1)  Y  -  bQ  +  +  b2X2 

where  Y  is  the  mean  reciprocal  response  time,  X  it  ths  potency  of 
anthrax  toxin*  in  log2  units,  and  the  b  values  are  regression  coefficients 

computed  from  ths  data  of  this  test,  The  values  of  ths  coefficient*, 
their  variances  and  covariance*,  are;  bQ  ■  -2, 991',  b^  ■  0,939; 

b2  ■  -0.031;  V(bQ)  ■  0.077121;  V(bj)  ■  0,009314;  V(b2)  ■  0.000068; 

■  -0.026902;  V(bQb2)  -  0.002238;  V^bg)  ■  -0,000800,  This 

rsgreseion  line  represent*  a  beets  upon  which  comparisons  of  potency  of 
anthrax  toxins  can  be  mads,  Thus,  test  toxins  can  bs  assayed  either 
indirectly  against  this  curve,  or  directly  with  parallel  assays  of  the 
reference  toxins. 

Development  of  procedures  for  indirect  assay  method,  To  use  the 
responses  of  120  rats  to  the  reference  toxins  [for  which  the  slope  of 
response  from  the  regression  data  (Figure  1)  has  been  calculated]  ,  wa 
recommend  use  of  the  Indirect  method  for  standardising  unknown 
potencies  of  anthrax  toxl  ns.  Ths  regression  was  nearly  linear  for 
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doses  from  16  to  128  units,  corresponding  to  response  times  from  240  to 
65  min,  Thus,  although  the  concentration  of  teat  or  unknown  toxins  Is 
arbitrary,  it  should  be  of  such  concentration  that  1  ml,  injected  lntrave- 
noualy,  will  kill  a  Fischer  rat  in  not  less  than  65  min,  nor  more  than 
240  min, 


Response 

Time 


Potency  Units 

Figure  1.  Regression  of  reciprocal  response  time  of  Fischer 
rats  on  log  dote  of  anthrax  toxins  expressed  in  potency  unite, 


To  test  the  potency  of  test  or  unknown  toxins,  tnough  animals  should 
be  used  so  that  ths  amount  of  variation  in  the  final  result,  that  can  be 
attributed  to  the  test  rats,  is  at  least  no  greater  than  the  amount  of  varia¬ 
tion  contributed  by  the  standard  rats.  Thus,  at  least  six  Fischer  rats  of 
200  to  300  g  from  a  suitable  colony  should  be  intravenously  inoeulatsd, 
three  with  2  ml  of  the  test  toxins,  and  three  with  1  ml, 
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The  test  is  based  on  the  mean  reciprocal  response  times  of  the  rats. 
(The  rat  response  is  very  uniform;  thus,  any  observed  nonresponse  must 
be  considered  the  result  of  technique  at  some  stage  of  the  assay  procedure.  ) 
This  is  simply  the  sum  of  reciprocal  times -to -death  of  the  rats  in  minutes 
(100/t)  with  the  average  time  calculated.  The  reciprocal  response  times 
of  the  rats  can  be  put  in  the  following  form; 

Reference  Toxins 
Y  =  100/t 


1  ml 

1.  _ 

Rat  2  .  _ 

3.  “ 
S  Y 
T  =  R, 


2  ml 


R,  +  R  = 


Test  Toxins 
Y  =  100/t 


1  ml 

1.  _ 

Rat  2. 

3.  “ 
SY  _ 

?  =  tl  _ 

T  +  T. 

i  t 


2  ml 

4.  _ 

5.  _ 

6.  _ 

SY  _ 

T„ 


where  R^,  R9 ,  T  ,  and  T ^  are  mean  reciprocal  response  times.  This 

form  for  calculation  can  be  used  for  either  the  direct  or  indirect  assay 
method. 


The  estimate  of  the  difference  in  potency  (D)  between  the  test  toxins 

and  the  reference  can  be  found  as; 

(T  +  T  )  -  (R  +  R  ) 

(2)  D  =  — - - - l _ L 

v  2L 
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from  the  table  above,  and  L  Is  the  average  slope  of  the  reference  dose* 
response  curve  at  the  two  dose  levels  used  In  the  test.  This  average  elope 
may  be  calculated  as: 

(3)  L  -  bL  +  b2  (Xl  +  X2) 

where  Xj  and  X2  are  the  dose  levels  of  the  reference  toxins  (in  log2 
potency  units)  that  were  used  in  the  test,  and  b^  and  b^  are  the  estimates 

of  the  regression  coefficients  from  equation  1.  When  the  test  is  run  using 
1-  and  2-ml  doses  of  toxins,  than  X^  ■  5  and  X2  ■  6.  Under  those  condi¬ 
tions  ■  0.92,  R2  ■  1,  34  from  equation  1,  and  L  ■  0,3983  from 
aquation  3,  so  that  equation  2  becomes: 

(T.  +  T,)  •  2.26 

(4)  D  -  - - 

where  the  letter  D  represents  the  amount  of  difference  between  the  test 
and  reference  toxins  in  terms  of  log2  potency  units,  If  D  is  positive,  then 

the  test  toxins  are  more  potent  than  the  reference,  whereas,  if  D  it 
negative,  the  test  toxins  are  less  potent  than  the  referenoe.  The  reference 
toxins  have  a  potency  of  3  logg  units  per  ml  at  a  concentration  of  lXj  thus, 

(he  potency  (P)  of  the  teat  toxins  in  log2  units  at  the  concentration  tested 
will  be  found  as: 

(3)  P  .  5  4  D 

To  find  the  number  of  potency  units  par  ml  of  the  test  toxins,  its 
potency  nebds  to  be  converted  from  log.,  units  to  logjg  units.  The  conver¬ 
sion  formula  is: 


log1Q  P  -  log2  P  logl0  2 
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The  value  of  P  in  unite  ie  found  by  looking  up  the  antilog  of  this  product. 
Thie  value  will  be  the  number  of  potency  units  per  milliter  of  the  test 

f  Awl  m  n  n  i  4- V.  _  ^  ^  .  _  ..  4  ..  .  i.  ]  .  i.  x  % 

•“****»•  atiUU  to  ■  bQU  i 

Estimation  of  variance.  There  is  variation  inherent  in  this  assay 
system  in  addition  to  the  variation  between  samples  of  toxins,  Thus, 
the  single  estimates  of  the  potency  of  any  particular  sample  of  an 
unknown  toxin  should  be  bounded  by  confident  limits.  To  determine 
these  limits  it  is  necessary  to  calculate  the  variance  (V)  of  the  estimate 
D  of  the  log.j  of  the  difference  in  potency  between  the  test  and  the  refer¬ 
ence.  The  variance  of  the  estimate  D  will  depend  on  the  variances  of 
the  observed  response  times  and  of  the  regression, 

If  we  express  D  as  N/G  where 

(6)  N  -  +  T2)  -  ^  +R2) 

and 

G  >  2L 

then  the  variance  of  D  can  be  expressed  as; 

(?)  V(D)  -  -L-  { V(N)  +  D2V(G)) 


which  will  apply,  becauso  N  and  G  are  estimated  from  independent  obsei 
vations  (Finney,  1952).  The  four  mean  reciprocal  response  times  are 
stochastically  independent;  thus,  the  estimate  of  V(N)  can  be  expressed 

as; 

(8)  V(N)  ■  V(Rl)  +  V(R2)  +  V(T1)  +  V(T2) 

where  V(T^)  and  V(T2)  are  obtained  directly  from  the  data  of  the  test, 
and  V(R^)  and  V(R2)  are  calculated  from  the  regression  line  as: 


Deaign  of  Experiment* 


2  38 


(9) 


V(R, )  =  VlY)  +  (X  -  XI2  Vfb .) 

1  ’1  *  i' 

+  (xt2  -  X2)2  V(b2), 


The  variance  of  G  i*  given  by  the  equation: 

V(G)  =  4{v(b1)  +  (XL  +  X2)2V(b2) 


(10) 


+  (x:  +  x2)  v(b1b2)}> 


When  the  teat  ia  run  uaing  1-  and  2 -ml  do*ei  of  toxin*,  then 
X^  =  5  and  X2  =  6.  Under  these  condition*: 

V(R  )  =  0,0134,  V(R2)  =  0.0018 

and 

V(G)  =  0.0355 

■  o  that: 

(11)  V(D)  »  {v(N)  +  0.  0355D2} 

and: 

(12)  V(N)  »  0.0134  +  0.0018  +  VfT^  +  V(T2)  , 

Example.  A  lample  of  toxins  of  unknown  potency  wn  tested  in  this 
laboratory.  It  wa*  known  to  kill  Fiicher  rat*  in  slightly  more  than  90 
min  when  injected  intravenously  in  doles  of  1  ml  at  a  concentration  of 
IX,  The  response  of  the  unknown  toxin*  was  compared  with  the  response 
curve  described  by  equation  1,  Each  of  three  Fiicher  rats  was  injected 
with  1  mi  of  the  teat  toxins,  and  their  reciprocal  response  times  in 
minutes  were  recorded  (Figure  2).  Three  other  Fiicher  rats  were  each 
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injected  intravenously  with  2  ml  of  the  test  toxins,  Their  reciprocal 
response  times  were  also  recorded  (Figure  2).  From  these  six  recip¬ 
rocal  response  times,  values  of  and  were  calculated,  Correspond¬ 
ing  values  of  and  were  obtained  from  the  regression  line  by 

substituting,  respectively,  the  values  5  and  6  for  X  in  equation  1,  The 
value  of  L  was  calculated  from  equation  3  by  use  of  the  values  5  and  £ 
for  and  X^.  The  values  5  and  6  were  used  in  these  two  cases,  because 

they  are  the  log^  of  the  number  of  units  in  1  and  2  ml  of  the  reference 
toxins . 

The  value  of  D  was  calculated  by  substituting  the  previously  calcu¬ 
lated  values  of  R^,  R^,  T^,  T^,  and  L  in  equation  2.  This  value  of  D 

was  found  to  be  0,  78.  This  indicates  that  the  test  toxins  were  9.  78  log^ 
unit  more  potent  than  the  reference.  A  1-ml  amount  of  the  reference 
toxins  contains  5  log^  units,  so  the  test  toxins  must  contain  5.78  log., 

units.  Thus,  the  test  toxins  have  55.  0  potency  units  per  ml  at  the  con¬ 
centration  testr-c.  (5.  78  X  .  301  =  1.  73978  log^  units), 

The  formulas  for  calculating  the  variance  of  the  estimate  D  of  the 
log2  of  the  difference  in  potency  between  the  test  and  the  reference  are 
described  above  as  equations  6  through  10.  These  calculations  were 
made  in  this  example,  and  it  was  found  that  SE  (D)  =  0.  26,  Using  normal 
theory,  the  95%  confidence  limits  of  D  become  UE(D)  =  1,  30,  and  LL(D) 

*  0,  26.  From  these  the  95%  confidence  limits  of  P  were  calculated  as 
UL(P)  =  79. 4  units  per  ml.  and  LL(F)  =  38.  0  units  per  ml. 

DISCUSSION.  Anthrax  toxins  are  composed  of  at  least  three  factors, 
I,  II,  and  111,  by  the  classification  of  Stanley  and  Smith  (1961,  1963)  or, 
respectively,  edema  factor,  protective  antigen,  and  lethal  factor  accord- 
-ir g-to  Beall_et_al.  (1962).  Both  in  vitro-produced  toxins,  as  used  in  this 
report,  and  in  vivo  toxins,  as  reported  by  Klein  et  al.  (1963),  may  be 
quantitated  accurately.  The  procedure  further  provides  an  effective 
reference  for  quantitating  natural  resistance  or  relative  immunity  as 
described  by  Klein  et  al.  (1963),  because  the  absolute  dose  of  toxins 
required  to  elicit  a  given  response  will  bear  a  definite  relationship  to 
host  resistance  or  susceptibility. 


Rtfirane#  Team 


Tttf  Tontft 


Y •  100/ t 
I  ml.  2  ml. 


Hat  {  2 
_  3 


Y  *  100/t 
I  ml.  2  ml 

i  _ Lil _ Li 


b.  •  -2.8911 


*  i  "  I  -  r  Bn  ■  «.  B viK 

:  :: : . 


I  Y _ Z  Y  3.79  4,82 

y  *«, _ tLii _ L2L_  v  •  t,  JJUnZLK 


R(  ♦  R2«  3.26 


V  V  2  .87 

_ '  ■  ■ 


VUjJ'  .07712091 
V  ( b!) »  .00951388 


I  y! _ rt^.ani  7,7308  V(b,J»  .00008104 

V(R1>_i0i24 - iSfiiL  V(T|>^00»8 - ,00U,  ^ 


l,V  W  V 


b,  »  0.9832 
b »  U  ♦  *  I*  0.3607 
*  0.3983 


—  (a,*  *j  )» _ li _ I*,  ♦  >U 

IT,  ♦  T  MR  ♦  R  ) 

o.— — U-i — L.  . 


I  L  .-LiilQ -  *•.  0.8084 

Rl’a-frW  Lt||  P  •  8  ♦  0.  8  °  - 


Logjo  P»  0  301  »  3.78  -  1.74  R*  33.0  U/ml] 


V(0)«  4{vib,)  ♦  («,♦«, I*  V(b,)*  («,♦«,)  V(  b,b,)}  • 

V(N)«  V(H,)^  V<R,i*  V(T,  I  ♦  VtT,  )  , 

V(D)a  { V4N)  ♦  0*  V  (0 )  1  , 

4L  »  1  1 


0.C333 


0.0211 


0.0672 


SC  (D) « 


UL  (0)  ■ 


LL<0). 


t09  UUP) 
10 


Lo9|o  LL<P)i 


UL(P)a  - I2il 


tio.  3.  Calculation  form  jut  putemy  of  anllfui  lotim, 


wmm  r*'****1 


243 


Design  of  Experiments 

\ 

The  biological  activities  of  these  compounds  are  numerous,  and  it 
is  likely  that  s'sjme  responses  are  still  to  be  discovered.  The  problem 
of  evaluating  activity  and  mode  of  action  of  compounds  which  have  a 
synergistic  biolo'gical  action  is  more  difficult  than  for  "single  compounds." 
Quantitation,  therefore,  is  important  to  allow  the  work  of  various 
investigators  to  be\related  more  exactly  to  each  other.  The  Fischer 
344  rats  are  commercially  available,  and  reference  anthrax  toxins  will 
be  provided  for  responsible  investigators  who  desire  to  work  with  this 
material  for  use  in  establishing  units.  The  methods  used  in  this  stand¬ 
ardization  of  these  toxins  may  be  appropriate  ‘to  the  standardization  of 
other  biologically  activ'c?  toxins. 
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SELF -DISPERSING  BOMBLETS- 

David  M,  Mom  and  Theodora  W.  Horner 
Boot'  Allen  Applied  Reaearch  Inc. 


ABSTRACT.  The  queation  haa  been  railed  concerning  the  lethal  haeard 
to  pereonnel  from  aelf-diaperaing  bombleta,  The  aolution  of  thia  queation 
involved  the  derivation  of  a  diatribution  and  the  computation  of  parameter  a 
for  a  apecific  problem.  The  baaic  method  uaed  waa  to  define  a  random 
variable,  0,  the  number  of  individual!  which  are  hit) 

N  n. 

0  =  E  (1  -  0  l)  , 

1-1 

where  N  ia  total  number  of  peraonnel  and  n.  ia  the  number  of  bombleta 
th 

atrikirg  the  i  individual.  The  moment-generating-function  of  thia  random 
variable  waa  found  and,  hence,  ita  diatribution  function.  The  diatribution 
of  caeualtiea  waa  found  to  be  Poleaon  under  the  general  aaeumptlona  of 
the  problem. 

The  queation  haa  been  raiaed  concerning  the  lethal  haeard  to  peraonnel 
from  aelf-diaperaing  bombleta  by  direct  hita.  In  trying  to  determine  the 
lethality  of  theae  bombleta  many  factora  muat  be  taken  into  account, 

Among  the  factora  which  bear  on  thia  problem  ia  that  of  protection. 

The  flight  of  the  bombleta  might  be  intercepted  by  treea,  buildinga,  or 
other  natural  or  man-made  obatructlona,  and  would  therefore  deacreaae 
the  chancea  of  a  lethal  hit.  In  thia  atudy  the  intereat  ia  directed  toward 
aaaeaaing  the  maximum  haeard  to  peraonnel.  It  la,  therefore,  aaeumed 
that  all  peraonnel  are  completely  expoaed.  It  ia  alao  aeaumed  that  all 
peraonnel  are  in  an  upright  pooition  and  no  peraon  providea  any  protection 
for  another  peraon.  Thua,  each  peraon  ia  completely  and  equally  expoaed 
to  the  poaeibility  of  a  direct  hit  by  a  bomblet, 

•Work  on  which  paper  ia  baaed  waa  aupported  by  contract  with  the  U.  S,  Army 
Biological  Laboratories,  Fort  Detrick,  Frederick,  Maryland. 
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Other  assumptions  made  in  order  to  assess  the  maximum  hazard  are 
that  all  personnel  are  within  the  target  area  of  interest  and  all  bomblets 
hit  somewhere  within  this  area.  It  can  also  be  assumed  that  the  vulner¬ 
able  portions  of  an  individual  are  his  head  and  neck.  If  other  portions 
of  the  body  are  struck,  it  is  assumed  that  lethal  damage  is  not  inflicted. 

The  objective  here  will  be  to  determine  the  hazard  to  personnel  on 
target  resulting  from  a  drop  of  self-dispersing  bomblets.  The  distribution 
of  the  number  of  lethal  hits  resulting  from  such  a  drop  will  be  determined 
and  in  addition  the  expected  number  of  such  hits  and  the  associated  vari¬ 
ance  will  be  found.  The  results  found  will  reflect  the  maximum  hazard 
involved. 

In  addition  to  the  theoretical  work  done  here,  the  results  for  a 
specific  case  will  be  given.  This  will  be  the  case  where  600  bomblets 
are  dropped  on  a  one  square  kilometer  area  which  contains  4000  persons. 

First  it  will  be  assumed  that  there  are  N  individuals  in  the  target 
area,  A^,.  There  are  n  bomblets  dropped,  all  of  which  land  in  the 

target  area.  Further  it  will  be  assumed  that  bomblets  and  individuals  are 
uniformly  and  independently  distributed  in  the  target  area;  however,  it 
will  be  shown  later  that  the  individuals  may  assume  any  distribution.  It 
will  also  be  assumed  that  individuals  and  bomblets  can  be  represented  by 
circles  with  areas  given  by 

(1)  A  =  -rrr  2 

P  1 

_-.li.V-4. 

(2)  Ab  =  <r2Z, 


where  r^  is  the  radius  of  the  critical  area  of  an  individual  and  these 

areas  for  all  individuals  are  considered  to  be  the  same,  and  r„  is  the 

radius  of  a  bomblet.  Now  in  order  to  produce  a  casualty,  the  center  of 
a  bomblet  must  fall  within  the  circle  with  radius 


(3) 


r  =  rl  +  r2 
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Casualty  Radius  Diagram 

The  target  area  can  be  divided  up  into  N  circular  cells,  each  with 
radius  r  ■  r^  +  r ^  representing  individuals,  plus  one  cell  which  repre¬ 
sents  that  part  of  the  target  in  which  there  are  no  individual’ s ,  We  can 
assign  a  value  p.  to  the  probability  that  a  bomblet  fall*  in  the  l**1  cell. 

•  fa 

Let  n^  represent  the  number  of  bomblsts  that  fall  in  the  i  cell.  Then 


N+l 

(4) 

£ 

i»l 

and 

N+l 

(5) 

Z 

i»l 

where  n  is  the  total  number  oi  bomblsts. 

The  interest  now  is  in  the  number  of  persons  hit  or  the  number  of 
casualties,  denoted  here  by  6  .  What  is  needed  is  a  variable  which  will 
give  the  number  of  casualties,  regardless  of  whether  an  individual  is  hit 
more  than  once.  One  such  variable  could  be  obtained  by  defining  a 
variable  which  is  either  aero  or  one  depending  on  whether  an  individual 
is  missed  or  hit.  If  such  a  variable  is  then  summed  over  all  individuals 
the  result  would  be  the  total  number  of  casualties,  6. 
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Note  that 


(6) 


and 


n. 
0  1 


1  when  n^  ■  0 
0  when  n^  >  0 


(7) 


0  when  n^  =  0 


1  when  >  0; 

rj 

that  is,  if  the  number  of  hits  of  an  individual  is  one  or  more,  (1-0'  ) 
will  be  one  and  will  be  aero  otherwise,  Thus,  let  us  define  our  variable 
of  interest  as 


(8)  6  ■  S  (1-0  *)  , 

1*1 

This  variable  tells  us  the  number  of  individuals  which  are  hit  and  it  is 
about  this  random  variable  that  we  want  more  information. 

Now,  before  going  on,  let's  look  more  closely  at  our  probabilities, 
where  (i  ■  1,2,  .  ,  ,  ,  N)  defines  the  probability  of  a  hit  of  the  individual 

in  the  i**1  cell.  Obviously,  the  probability  that  any  particular  bomblet 
hits  any  particular  individual  is  the  same  for  all  bomblets  and  all  indi¬ 
viduals.  Also  it  is  quite  clear  that  the  probability  of  a  randomly  chosen 
bomblet  from  a  uniform  distribution  of  bomblets  hitting  any  individual  is 
uqual  to  the  ratio  of  the  area,  A  ,  of  the  circle  with  radius  r  to  the  total 
target  area,  A^. 


Thus 

(9) 


P ■  ac/aT' 


*  ‘  T-'i' 


Design  of  Experiment* 
where 


Ac  ‘  ff(rl  +  r2>  ‘ 


Note  that  aince 


(11)  E  p.  »  1 

i»l  1 

and  the  ,  i  =  1, 2 ,  ,  ,  ,  ,  N,  are  equal,  we  thue  have 

(12)  PN  +  1  ■  1  -  Np  . 

What  we  have  1*  essentially  the  probability  of  a  randomly  selected 
point  being  within  a  certain  area.  Note  in  Figure  2  that  the  probability 
that  a  randomly  selected  point  lies  in  a  given  circle  is  the  same  in  A 
and  B  and  also  that  the  probability  of  at  least  x  of  the  n  points  lying 
within  a  circle  is  the  same  in  both.  Based  on  this  it  can  be  seen  that  our 
results  will  be  independent  of  the  distribution  of  personnel. 


Figure  2 

Possible  Personnel  Configurations 
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Now  let  us  look  at  an  analogous  situation. 


....  „ ^ ^ ~ i . .  «k... 


_  kaU, 


Suppose  that  we  have  N  +  1 


Table  I 

Distribution  of  Balls  Falling  into  Cells 


Ceil 

Probability 

Number  of  Balls  Falling 
Into  Cell 

1 

a  P 

nl 

2 

P2  -  P 

n2 

N 

a 

■ 

Z 

o. 

• 

"N 

N 

N  +  1 

"N+l  "  ‘-Np 

■w "  ■  "i 

This  Is  a  multinomial  situation  where 


(13)  n2>  ....  n^j+1)»  n!  /  n  n^1 

i*l 


N  +  1 


"l  n2 

Pi  P2 


•  P 


N+l 


N+l 


Since  it  is  9  in  which  we  are  interested,  we  need  to  discover  the  distri¬ 
bution  of  9 .  The  approach  taken  here  will  be  to  find  the  moment-generat 
ing-function  of  8  and  from  it  the  distribution  of  6. 
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Recalling  the  definition  of  moment- generating-function  from 
mathematical  statistics  and  substituting  tor  e  from  equation  (o),  we 
have 


(t)  =  E 


Now 

(15) 


whan  rij  ■  0 


whjn  n^  >  0, 


and  equivalently 

ni  n 

(16)  a"*0  "1  +  0  *(e"l-l). 

Note  that  (16)  holds  Identically  and  that  the  right  hand  side  is  not  part 
of  a  series  expansion,  Substituting  back  in  (14),  we  havs 


\ 


'  *”••-**  -v 


Design  of  Experiments 


2  54 


(17) 

Now  let 

(18)  b.  =  Oi(e't-l) 
and  substitute  in  (17): 


(19) 


M0(t)  =  etNE 


N  1 

n  (i+V> 

i=l  I 


+  11  b,b. 
i  j  J 

J>  ' 


+  SEE  b.b.b,  + 
i  jk  1  ‘ k 
k>  J>  1 


+  I  Z  ...  I  b.b..  .  .  b 
i  j  m  iJ  m 

m>  ...  >  j  >  i 

Now  taking  the  expectation  of  a  typical  term,  say  the  g  +  l-t  term  and 
substituting  from  (18),  we  have 
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(20) 


.e  i  ...  r  s(b1b...bj 


f  n  n  n 

E  E  .  .  .  E  E  <(e't-l)g0  lQ  2.  .  .0  8 


(e’*-!)8  2  E  ...  E  E  *(0  ‘0  fc...O 


n  n  n 
2  n  8 


Now  the  expectation  of  the  last  factor  in  (20)  is 


n.  n„  n 


"l  *2  n- 


E  (0  *0  2, . . 0  8  )  ■  E  0  l0  Z.  .  .  0  gf  (nr  ng . n^) 


which  becomes  upon  substitution  from  (13) 


nl  n2  Y 

Z  \Q  0  .  .  .0  * 


n.  n„  n  ^  n2 


=  e  o  Jo  2...o  8  Pl  pa  ...  PN 


■  1  (o-p/'lop/2  •••  to- p  )  « 


(21) 


N+l 
'  '  PN+1 


0  +  °  +  ...  +0+p  +PN+P 


N  rN+l 


The  maximum  value  of  gp  ia  Np,  However,  Np  ia  extremely  am&ll 
aa  aeen  from  the  example  following  the  theory,  Since,  therefore,  gp 
ia  extremely  email, 
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(l-gp)n=  «"X1P8  i 


which  follow*  because 


,-ngP  =  (e-gp}n 


=  0-gP+^  -  ^  +  ...  )n 


*  (1  -  gp)n 


Therefore 


Me(t) 


g-o  / 

tN  T  N\  r  -np,  -t  n  1  g  i .  »N * | 

-  e  I  e  r(e  -1)  (1) 

grO\*j  L 

» .tN  [.-nv'-D  +  i]N 


t  -np/  -t  ,  t 
■  e  e  r(e  -1)  +  e 


-np  ,,  t>  t  ^ 
■  «  (1-*  )  +  « 

.  •> 


-np  t  -np  ,  t 

e  -ee  +e 


-np  ,  t  /.  -np. 

e  r  +  «  (1  -  e  r) 


’  A  1  Jvt 
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Now  in  the  above  result  Let 

(28) 

and 


We  then  have 

(29)  Mfl(t)  es  (Q  +  Pe*)N 

which  can  be  recognised  ae  the  moment- generating-function  for  the  binomial 
distribution,  Thus  9  is  approximately  binomially  distributed  with  param¬ 
eters  P,  Q,  and  N.  The  expected  value  of  9  or  the  mean  number  of 
casualties  is  given  by 

E  (e)  «  NP 

(30)  =  N  (1  -  e‘np) 

.  2  3 

■  N  1  -  (1  -  np  +  +  ■  •  • ) 

M  Nnp, 

the  last  step  following  since  np  is  extremely  small.  Thus  the  E(e)  is 
small  unless  N  is  extremely  large.  Also  because  P  is  small,  the 
distribution  of  0  can  be  approximated  by  a  Poisson  distribution  and 
therefore  the  variance  is  also  approximately  Nnp.  The  distribution  of 
6  .  where  9  is  the  number  of  casualties,  is  given  by 

(31)  p(9)  =  (Nnp)®  e’Nnp/9.' 


Now  let's  look  at  the  specific  problem:  namely  that  of  dropping  600 
bomblets  on  a  one  square  kilometer  target  which  contains  4000  personnel. 
It  is  given  that: 
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a.. 

A 

in10 

a  1 0  rm 

1 

b. 

A 

=  314  cm2 

P 

c. 

N 

=  4  x  103 

d. 

n 

=  6  x  102 

e. 

ri 

-  10  cm 

f. 

r2 

s  7  cm  , 

From  these  it  is  found  that 
p  =  ac/at 

■  it  (10  +  7)2  /  1010 

»  9.1  x 10*8  , 


and  that 

E  {©}  ■  Nnp 

«  (4  x 103) (6  x 102) (9.1  x IQ"8) 


-  0.22 


and 

VAR  (e)  ■  0.22. 

Note  that  N  ,  which  is  the  maximum  value  of  gp,  is  Np  ■  3.  64  x  10 , 
P 

a  very  small  quantity.  Further,  it  is  found  that  the  probability  of 
exactly  x  casualties  under  the  given  assumptions  are  as  in  Table  11, 
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Table  II 


Caeualty  Distribution 


Number 

Probability 

of 

of 

Casualties 

Occurrence 

0 

0.80252 

1 

0. 17655 

2 

0,  01942 

3 

0.  00142 

4 

0, 00006 

5 

0,  00000 

Note  that  the  expected  number  of  casualties,  0,22,  is  approximately  0.  0055 
percent  of  the  4000  personnel  or  approximately  one  casualty  in  five 
similar  drops. 

The  haxard  to  personnel  resulting  from  a  drop  of  self-dispersing 
bombleta  was  found  to  be  very  low.  It  was  found  that  the  number  of 
casualties,  6,  is  Poisson  distributed  of  form 

p  (0)  =  (Nnp)6  e'Nnp/0  , 


where  N  is  the  number  of  personnel  on  target,  n  is  the  total  number 
of  bombleta ,  and  p  is  the  probability  that  an  individual  is  hit  by  a 
particular  bomblet.  For  the  specific  case  of  600  bombleta  and  4000  per¬ 
sons  in  a  one  square  kilometer  area,  p  is  approximately  9.1  x  10*®  and 
the  expected  number  of  casualties  is  0.  22, 


EXPLOSIVE  SAFETY  AND  RELIABILITY  ESTIMATES 
FROM  A  LIMITED  SIZE  SAMPLE 

J.  N.  Ayres,  L.  D,  Hampton,  I.  Kabik 
U.  S.  Naval  Ordnance  Laboratory 
White  Oak,  Silver  Spring,  Maryland 


ABSTRACT.  The  problem  of  predicting,  from  amall  sample  testing, 
high  reliability  and/or  high  safety  for  explosive  items  is  becoming  more 
acute,  Often  the  available  test  sample  is  no  greater  than  200,  Only  a 
single  test  per  item  is  allowable  and  the  data  is  always  of  the  go/no-go 
variety.  Methods  being  used  for  making  conservative  extrapolations  to 
the  high  and  low  probability  of  firing  points  are  reviewed  and  illustrated, 

The  question  of  how  to  do  the  job  better  is  posed  and  left  to  the  clinicians 
for  answer. 

INTRODUCTION.  The  problem  which  we  wish  to  present  is  how  to 
make,  with  small  samples,  reasonable  estimates  of  the  stimuli  correspond¬ 
ing  to  the  high  and  low  probability  of  firing  of  electro -explosive  devices 
(SCO's). 

A  typical  EED  is  shown  in  Fig.  1,  Essentially,  it  consists  of  an 
insulator  carrying  two  electrical  conductors  across  which  is  attached  a 
resistance  wire.  Surrounding  the  resistance  wire  is  a  sensitive  explosive. 
When  electrical  energy  is  dissipated  in  the  wire,  the  resultant  temperature 
rise  causes  the  explosive  to  heat  and  react  chemically,  and  thus  produce  an 
explosion. 

EED's  are  used  by  the  military  for  a  number  of  purposes;  to  cause 
detonation  of  explosive  loaded  shells,  bomba,  grenades,  missiles,  mines, 
etc.  ,  to  ignite  propellants  for  guns  and  rockets,  to  close  switches  such 
as  in  fuse  arming  circuits,  to  release  stores  from  aircraft,  to  eject  pilots 
from  aircraft,  and  to  separate  missile  stages.  These  are  only  some  of 
the  more  common  uses. 

The  designer  of  explosive  ordnance  has  always  been  faeed  with  the 
problem  of  estimating  the  safety  and  reliability  of  his  explosive  system. 

The  safety  and  reliability  associated  with  the  EED  of  electrically  operated 
explosive  ordnance,  are,  of  course,  important  links  in  this  system.  For 
reasons  to  be  given,  estimating  the  safety  and  reliability  to  be  expected 
from  an  EED  cubjected  to  various  stimuli  is  usually  not  simple.  The 
ordnance  designer  in  the  past  has  often  overcome  lack  of  information  on 
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reliability  at  least,  by  the  numbers  of  items  strategically  used,  i.  e.  ,  the 
number  of  shells  fired  or  the  number  of  bombs  dropped,  etc.  Thus 
unreliability  could  be  compensated  for  in  actual  field  usage. 

Modern  weapons  and  warfare,  however,  have  introduced  new  problems. 
It  is  too  costly  to  fire  large  numbers  of  expensive  ordnance  devices;  the 
catastrophic  results  of  a  safety  failure  of  certain  types  of  munitions  are 
intolerable;  the  intensity  of  certain  stimulis  which  may  cause  inadvertent 
firing  (electro-magnetic  radiation  from  radars  for  example)  has  increased 
tremendously  in  the  last  decade  and  is  slated  to  increase  further.  These 
changes  have  made  it  virtually  mandatory  that  reasonable  estimates  of 
response  of  EED's  to  electrical  stimuli  be  made. 

RELEVANT  FACTS. 

(a)  For  economic  reasons  it  is  impossible  to  make  a  direct  demon¬ 
stration  of  the  response  of  interest.  The  stimulus  for  reliability  of  99.  9+% 
is  usually  desired  at  95%  confidence.  Conversely,  safety  may  demand 
estimates  at  95%  confidence  of  the  stimulus  at  which  no  more  than  1  in  a 
million  devices  would  be  expected  to  fire.  Funds  are  never  available  to 
run  direct  demonstration  tests. 

(b)  The  nature  of  EED's  preclude  repeated  testing  on  a  single  device. 
Since  these  systems  respond  chemically  to  temperature  elevation  at  the 
resistance  wires,  it  is  not  known,  once  a  single  test  at  a  given  stimulus 
was  large  enough  to  have  altered  the  EED's  response  characteristics.  It 
must  therefore  be  assumed  that  the  possibility  of  alteration  is  great  enough 
to  preclude  more  than  one  test  on  a  given  EED.  The  only  piece  of  informa¬ 
tion  thus  possible  from  each  single  test  is  either  the  EED  fired  or  failed 

at  that  particular  test  stimulus. 

(c)  It  has  been  found\  from  a  large  number  of  firings  on  EED's 
(approx.  10,000  firings  of  Squib  Mk  i),  that  no  standard  distribution 
function  fits  exactly  the  tails  of  the  observed  EED  stimulus -response 
distribution.  A  number  of  distribution  functions  have  been  tested  for  their 
conformance  to  the  experimental  firing  data.  They  all  fail  at  the  tails 

of  the  curve,  see  Fig.  2.  But  it  is  precisely  these  regions  of  the  distribu¬ 
tion  which  we  must  estimate. 


Design  of  Experiments 


263 


(d)  Usually  no  more  than  200  test  samples  are  available  to  make 
estimates  on  one  side  of  the  mean  firing  (507o)  point,  whether  high  or  low, 
Even  a  sample  sise  of  200  is  sometimes  very  difficult  to  obtain  and  may 
be  quite  expensive, 

(e)  Popular  test  schemes,  such  as  the  "Bruceton"  test^,  which  are 
conservative  of  sample  sise,  often  give  poor  estimates  because  of  long 
extrapolation,  poor  estimate  of  the  standard  deviation,  and/or  non-appli- 
cability  of  the  selected  underlying  distribution^'^, 

THE  PROBLEM,  By  now  it  should  be  obvious  that  we  must  make 
multi-million  dollar  estimates  on  tens  or  hundreds  of  dollars  worth  of  data. 
We  must  design  our  experiments  so  that  we  most  wisely  expend  our  avail¬ 
able  samples  so  that  we  can  minimise  the  error  of  maxing  extrapolations 
to  the  desired  answer.  We  realise  that  extrapolation  is  at  best  a  risky 
business  but:  is  there  any  uther  choice? 

iin  the  following  section  we  will  tell  you  what  we  think  we  know  and 
the  methods  we  are  now  using, 

The  basic  problem  is  to  collect  data  which  will  permit  the  computation 
of  the  variation  of  the  probability  of  firing  ae  a  function  of  the  firing 
stimulus.  It  is  dssirable  to  allocate  the  samples  so  that  the  data  collected 
will  be  as  close  as  posslbls  to  the  functioning  level(s)  we  wish  to  estimate. 
Ideally  we  should  collect  go/no-go  data  at  a  number  of  stimulus  levels. 

As  shown  in  Fig,  3,  we  wish  to  estimate  the  stimulue,  Xe,  at  which  w* 
can  expect  a  high  level  of  response,  Ye,  -  We  show  data  collected  at  five 
levels  of  stimulus  X^,  X^,  .  .  X&,  A  line  has  been  fit  to  the  observed  data 

and  at  point  Xe,  Ye  on  this  line,  is  the  intersection  which  gives  ue  the 
desired  stimulus  value. 

The  process  of  drawing  the  straight  line  shown  in  Fig.  3,  and  making 
the  indicated  prediction  Implicitly  makes  the  following  aseumptione. 

1.  That  there  is  no  sampling  error, 

2.  That  the  distribution  function  is  chosen  correctly,  and 

3.  That  there  is  no  systematic  error  in  the  instrumentation 
oi'  testing  procedure. 
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But  we  know  that  there  must  be  some  sort  of  error  simply  because  the 
uata  points  do  not  fall  on  the  line.  By  performing  the  "Chi-Square" 
statistical  test  on  the  data  we  can  decide  whether  or  not  the  observed 
variability  (scatter)  is  what  might  be  expected  from  sampling  error  alone. 
If  this  is  the  case,  then  we  can  draw  an  appropriate  confidence  band  as  in 
Fig.  3. 


WHAT  WE  HAVE  DONE.  But  rather  than  multi-point  testing  we  have 
made  what  we  believe  to  be  conservative  estimates  of  extreme  probability 
of  firing  points  by  the  test  and  extrapolation  procedures  given  below. 

To  minimize  the  importance  of  assumptions  regarding  the  frequency 
distribution  it  is  again  desirable  to  base  these  estimates  on  data  taken  as 
close  as  possible  to  the  per  cent  point  to  be  determined.  The  simplest 
such  test  would  be  one  which  calls  for  testing  at  two  stimulus  levels  near 
the  region  in  question.  One  of  the  two  levels  will  be  farther  from  the 
mean  and  closer  to  the  desired  point  than  the  other.  This  will  be 
designated  the  remote  stimulus  level.  The  data  obtained  can  then  be 
extrapolated  to  determine  the  stimulus  associated  with  the  desired  per 
cent  point,  In  planning  such  an  experiment  the  following  conditions  should 
be  met; 


a.  The  difference  between  the  stimuli  used  should  not  be  small 
compared  to  the  extrapolation  distance  (the  difference  between 
the  desired  point  and  the  observed  remote  stimulus). 

b.  The  number  of  trials  at  the  remote  stimulus  level  and  the 
expected  response  at  this  level  should  be  chosen  so  that  the 
probability  of  observing  a  saturated  level  (either  all-fires  or 
all  fails)  is  small*. 

c.  The  number  of  trials  made  at  the  remote  functioning  level 
should  be  greater  than  the  number  of  trials  at  the  level  closer 
to  the  mean  in  an  attempt  to  obtain  equal  weighting  of  the  two 
levels,  A  good  choice  is  to  take  the  number  so  that  the  product 
np  (1-p)  is  the  same  for  both  levels,  where  n  is  the  number  of 
trials  and  p  is  the  expected  probability  of  fire. 


#If  a  saturated  level  is  observed,  one  trial  can  be  converted  to  l/ 2  fire 
+  1/2  fail.  Or  another,  reversed,  trial  can  be  arbitrarily  added  to  the 
data,  Either  method  will  give  a  conservative  result. 


vc  a* 


f  Experiments 


265 


It  is  assumed  that  only  two  hundred  Bampl.es  are  available  to  estimate 
either  an  extremely  high  or  else  an  extremaly  low  probability  of  firing. 
The  general  procedure  will  be  illustrated  below  for  a  high  probability 
point;  a  numerical  example  is  given  in  Appendix  A. 

a.  Run  a  preliminary  Bruceton  type  test  on  20  samples  using 
a  log  transform  for  the  dosage*. 

b.  Use  the  Bruceton  results  to  e  stimate  the  X+0,  2s  ,  X+0 . 4s , 
and  X+l,  3s  level3‘,“. 

c.  Test  50  EED's  at  the  computed  X+0, 4s  level. 

d.  If  more  than  5  fail,  test  130  samples  at  the  above  calculated 
X+l.  3s  level, 

e.  If  5  ox  itwer  failures  occur,  continue  testing  until  130 
samples  have  been  tested,  and  test  50  at  the  calculated 
X+0.  2s  level. 

5 

f.  Using  a  log-logistic  probability  space,  plot  the  two  points. 

g.  Extrapolate  the  straight  line  through  the  points  so  obtained 
to  the  desired  probability  or  stimulus  value. 

By  using  only  two  points  we  have  no  way  of  applying  the  chi- square  test. 
Nor  can  we  draw  the  confidence  band  without  a  further  assumption,  To 
obtain  more  conservatism,  two  methods  have  been  used. 

Heterogeneity  Assumption 

We  proceed  as  above  but  assume  a  heterogeneity  factor4"*1*  of  1  in  the 

equation  for  the  confidence  limit.  This  assumption  allows  computa¬ 
tion  stnd  drawing  of  the  confidence  band  as  in  Fig,  4,  Implicit 


^Considerable  testing  has  led  us  to  believe  a  logarithmic  dosage  to 
stimulus  transform  is  of  proper  form, 

**For  low  probability  estimates  these  terms  would  be  X-0,  2s ,  X-0.4S, 
X-l.  3s,  and  following  computations  would  be  consistent, 

#++  2 

F  »  ,  where  F  =  heterogeneity  factor  and  n  ■  the  number  of  test 

1 _ 1  - 


i 

i 


s 
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in  the  assumption  are  the  assumptions  previously  given  also,  i.  e.  , 
we  have  chosen  the  correct  distribution  function;  there  is  no 
systematic  error  in  the  instrumentation  and  test  procedure;  and 
only  normal  sampling  error  occurs. 

Binomial  Method 


Using  the  second  method  of  gaining  conservatism,  rather  than 
plotting  the  measured  points  directly,  calculate,  at  a  desired 
confidence  level  (say  7  5%),  the  one-sided  lower  value  of  the 
higher  percentage  firing  point,  and  the  one-sided  upper  value 
of  the  lower  percent  firing  point.  Plot  these  points  in  a  log- 
logi  stic  probability  space.  Draw  the  straight  line  through 
these  points  and  extrapolate  to  the  desired  value.  Sfce  Fig.  5, 

It  is,  of  course,  possible  that  if  too  conservative  a  value  be  set  for 
the  confidence  limits  of  the  upper  one-sided,  lower  and  the  lower  one¬ 
sided,  higher  per  cent  firing  points,  the  slope  of  the  line  drawn  through 
these  limits  will  be  negative,  Such  a  situation,  when  it  occurs,  is  not 
realistic  and  this  more  conservative  estimating  technique  should  be  aban¬ 
doned. 

Our  experience  has  shown  us  that  although  the  logistic  distribution 
function  does  not  give  an  accurate  fit  to  EED  distribution  functions  at  the 
tails,  it  at  least  errors  cn  the  conservative  side,  i.e.  ,  it  will  predict  a 
lower  safety  than  actually  exists  and  a  lower  reliability  than  actually 
exists. 

The  two-level  test  and  analysis,  then,  is  one  technique  which  we  have 
used  to  make,  with  limited  samples,  estimates  of  extreme  probability  of 
firing  points.  We  could  certainly  devise  more  elaborate  and  sophisticated 
variations,  but  we  wonder  if  those  more  skilled  than  we  in  statistical 
theory  might  not  be  able  to  recommend  alternate  procedures  which  can 
do  the  job  better.  More  specifically,  we  have  wondered  about,  and  have 
planned  to  work  on,  the  application  of  non-parametric  statistical  methods 
to  the  problem.  The  clinic's  opinion  and  advice  on  this  matter  could  be 
beneficial  since,  at  the  time  of  this  writing  (June  1964),  we  are  only  in 
the  preliminary  thinking  stage. 

Finally,  we  have  been  hopeful  that  some  combination  might  be  made 
of  statistics  and  the  underlying  physics  of  the  mechanism  by  which  wire 
bridge  EED's  function,  to  put  bounds  or  the  degree  of  extrapolation 


Design  of  Experiments 


267 


needed  in  making  our  estimates.  In  this  regard  our  work  has  shown  that 
the  heating  of  a  wire  bridge  ZED  can  be  represented  by  the  mathematical 
equation: 

Cp  St  +  y0  B  p{t) 

where  -  heat  capacity  of  bridge  plug  explosive 

8  =  temperature  elevation  above  ambient 

t  =  time 

y  =  heat  loss  factor,  and  p(t)  =  power  input. 

6  7 

The  combination  of  this  equation  ’  with  Bowden's  hot  spot  theory  of 
explosions^  has  led  to  fai-ly  accurate  representation  of  EED  firing 
characteristics  over  a  limited  range  of  input  times  (i.  e.  ,  average  pow. 
ers).  Since  equipment  is  available  for  making  independent  measurements 

of  C  ,  y,  and  C  /y ,  the  cooling  time  constant,  it  appears  possible  to 
P  P 

measure,  on  individual  EED's,  parameters  which  should  be  directly 
related  to  their  individual  firing  characteristic*.. 
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APPEND  DC  A 

ILLUSTRATIVE  EXAMPLE 


The  units  used  for  X  ere  in  terms  of  the  transformed  variable. 

The  twenty  trial  Bruceton  gave  a  mean  of  20.  314  and  standard, 
deviation  of  0.  589. 

The  two  test  levels  are  then 

m+0,  4a  =  20.  55 
m+1.  38  =  21.  08 . 

The  results  at  these  levels  were 

Near  level  35/50  *  70% 

Remote  level  113/130  ■  86.92%. 

The  upper  95%  confidence  limit  at  the  near  level  is  78.  68%.  The 
lower  95%  confidence  limit  at  the  remote  level  ie  81.  94%. 

A  straight  line  through  the  observed  points  is 

Y  «  1.13019  X  -  22. 7014 
(Y  in  Normits). 

This  gives  estimates  as  follows: 

95%  point  21,  542 

99%  point  22.144 

99.  99%  point  23.347. 

The  equation  for  the  lower  95%  confidence  band  assuming  the 

heterogeneity  factor  to  be  unity  is 


Y  *  1. 13019  X  -  22.  7014-1.  645  Vo7oi4909+0.  213434(x-20.  857)2 
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This  gives  estimates  as  follows; 

95%  point  22.  9 

99%  point  24.  8 

99.  99%  point  29,  0  . 

The  straight  line  through  the  binomial  limits  on  the  observed  points 
has  the  equation 

Y  =  0.  2226  X  -  3,  7784  . 

This  gives  the  following  estimates 

95%  point  24.37 

99%  point  27.43 

99.  99%  point  33,69. 

Using  the  same  data  with  the  logistic  assumption,  we  have  the  follow¬ 
ing  analysis 

at  the  near  level  35/50  «  70% 

L  =  In  ||  =  0.  8473 

at  the  remote  level  113/130  =  8‘.  92% 

L  a  In  =  1,  8942  . 

The  straight  line  through  these  points  is 
L  =  1.  975  X  -  39.  7447  , 

This  gives  the  following  estimates 


L  X 

95%  2,9444  21.6 

99%  4,5951  22,4 

99.99%  9,2102  24.8 


The  binomial  confidence  limits  as  before  are 


near  level  1.  306; 


remote  level  1,  512 
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The  straight  line  through  these  points  is 
L  =  0.  389  X  -  6.  688 
which  gives 

95%  point  24.  8 

99%  point  29.  0 

99.  99%  point  40,9. 

The  hyperbola  for  the  lower  95%  confidence  band  has  the  equation 

L  =  1.  9753  X  -  39,  7447  -  1.  645^ 0,  039557+0.  579926(X-21.  86)^ 

which  has  an  asymptote 

L  =  0.  723  X  -  13. 6224  . 

Estimates  are 

95%  point  22.  9 

99%  point  25.  2 

99.  99%  point  31.6 

Summary  of  these  calculations  results 


Normal 

Logistic 

Straight 

Line 

95%  conf, 
band 

Binomial 

Straight 

line 

Conf, 

Band 

Binomial 

95%  point 

21.  54 

22.  9 

~  24.4 

ITT 

22.  9 

l4.~8  ““ 

99%  point 

22. 14 

24.  8 

27,4 

22.  4 

25.  2 

29.  0 

99.  99%  point 

23.  35 

29.  0 

33.  7 

24.  8 

31.  6 

40,9 

Comparison  of  these  values  shows  the  more  conservative  nature  of  the 
logistic  distribution.  The  difference  is  not  marked  at  the  95%  point  but 
does  show  up  at  the  more  extreme  points. 
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CYCLIC  DESIGNS* 


H.  A,  David  and  F.  W.  Wolock 
University  of  North  Carolina  at  Chapel  Hill  and  Boston  College 


1.  INTRODUCTION,  Cyclic  designs  are  incomplete  block  designs  con¬ 
sisting  in  the  simplest  case  of  a  set  of  blocks  obtained  by  cyclic  develop¬ 
ment  of  an  initial  block.  More  generally,  a  cyclic  design  consists  of 
combinations  of  such  sets  and  will  be  said  to  be  of  size  (n,  k,  r),  where 
n  is  the  number  of  treatments,  k  the  block  size,  and  r  the  number  of 
replications. 

It  is  well  known  (e.  g.  Bose  and  Nair  [2]  )  that  cyclic  development  of 
a  suitably  chosen  initial  block  is  one  of  the  methods  of  generating  designs 
with  a  high  degree  of  balance  in  the  arrangement  of  the  treatments  such 
as  balanced  incomplete  block  (BIB)  designs  and  partially  balanced  incom¬ 
plete  block  designs  with  two  associate  classes  (PBIB(2)  designs),  Again, 
the  cyclic  type  is  a  rather  junior  partner  among  the  five  types  into  which 
Bose  and  Shimamoto  [3]  classify  PBIB  (2)  designs.  The  emphasis  in 
these  and  many  related  papers  has  been  understandably  on  the  number  of 
associate  classes,  the  cyclic  aspect  being  incidental.  In  the  preeent 
article  we  proceed  in  opposite  fashion  putting  the  cyclic  property  first, 

It  will  be  shown  how  cyclic  designs  may  be  systematically  generated  and 
how  the  non-isomorphic  designs  of  given  size  may  be  enumerated  and 
constructed.  All  such  designs  are  PBIB  deslgne  but  may  have  up  to 
jn  aesociate  classes.  For  n  jg  15  and  k  ■  3,  4,  5,  tables  of  the  most 
efficient  cyclic  designs  are  presented  and  comparisons  with  BIB  and 
PBIB  (2)  designs  are  made. 

Points  which  make  cyclic  designs  attractive  are! 

(i)  Flexibility.  A  cyclic  design  of  size  (n,  k,  ik)  exists  for  all  positive 
integers  n,  k,  i  .  If  n  and  k  have  a  common  divisor  d  then  e  "frac- 
ional  set"  of  size  (n,  k,  k/d)  exists  corresponding  to  each  d, 
Fractional  sets  may  be  combined  with  designs  of  size  (n  k,  ik)  to 
form  fresh  designs,  or  used  by  themselves  especially  if  n  is  large, 
Thus  there  are  cyclic  designs  for  many  sizes  (n,  k,  r)  for  which  no 
PBIB  (2)  design  is  available,  but  the  reverse  may  also  happen. 


"Research  supported  by  the  Army  Research  Office,  Durham,  and  the 
National  Institutes  of  Health.  This  paper  has  been  submitted  for  publi¬ 
cation  in  the  "Annals  of  Mathematical  Statistics.  " 
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(ii)  Eo-cc  of  tcjjt caencation.  ino  plan  of  the  experimental  layout  is  needed 
since  the  initial  block  or  blocks  suffice, 

(iii)  Youden  type,  In  view  of  their  method  of  generation  cyclic  sets  with 

r  =  k,  and  hence  combinations  of  such  sets,  provide  automatic  elimina¬ 
tion  of  heterogeneity  in  two  directions, 

(iv)  Analysis ,  For  cyclic  designs  the  coefficient  matrix  of  the  normal 
equations  is  a  circulix,  The  inverse  matrix  may  therefore  be  obtained 
explicitly  (as  another  circulix),  thus  making  possible  a  general  method 
of  analysis,  Questions  of  analysis  will  not  be  considered  further  here 
since  methods  given  in  a  special  case  by  Kempthorne  [9]  continue  to 
apply  with  minor  modifications.  However,  details  and  aids  to  analysis 
are  presented  in  [12]  . 

Cyclic  designs  as  a  class  in  their  own  right  were  introduced  for  k  **  2 
by  Kempthorne  [9]  and  Zoellner  and  Kempthorne  [13]  ,  Design  aspects  for 
the  case  k  =  2,  which  has  some  special  features,  were  considered  in  [6] 
and  [7]  ,  and  will  not  be  trr'  ?d  in  this  paper,  For  general  k  cyclic  designs 
are  closely  related  to  the  circi’  .r  designs  uf  Das  [5]  .  See  also  ths  survey 
of  non-orthogonal  de«ign~  >  y  earce  [ll]  who  .a’iS  cyclic  designs  s  "little 
publicized  class.  "  PBI5  designs  have  been  studied  from  an  algebraic  point 
of  view  in  a  series  of  papers  by  Masuyama,  In  some  of  these  (e.  g.  [10]) 
reference  is  made  to  cyclic  designs  but  no  detailed  results  are  obtained. 

2,  CYCLIC  SETS,  Label  the  treatments  0,  1,  2,  ,  ,  ,  ,  n-1.  To  fix  ideas 
consider  the  arrangement  of  n  ■  7  treatments  in  blocks  of  sice  k  ■  3,  The 
complete  design  of  (^)  a  35  distinct  blocks  may  be  set  out  as  follows; 


[012] 

012 

123 

2  34 

345 

456 

560 

601 

(013) 

013 

124 

235 

346 

450 

561 

602 

(ol  4  j 

014 

125 

236 

340 

451 

562 

603 

(01 5] 

015 

126 

230 

341 

452 

563 

604 

(024) 

024 

135 

246 

350 

461 

502 

613 

F rom  any  block  the  others  in  the  same  row  may  be  obtained  by  increasing 
each  object  label  in  turn  by  1 ,  2,  3,  4,  5,  6,  and  reducing  modulo  7 .  The 


Design  of  Experiments 


285 


rows  have  been  arranged  to  start  with  the  block  of  lowest  numerical  value 
and* are  designated  by  the  initial  block  placed  in  braces.  We  call  each  row 
a  cyclic  set. 

A  block  may  also  be  conveniently  represented  by  identical  beads  spaced 
regularly  on  a  circular  necklace.  Fig.  1  shows  the  blocks  012  and  123. 


Figure  1 

The  set  {^012}  is  then  generated  by  successive  unit  rotations. 

It  is  not  difficult  to  show  that  each  cycle  set  forms  a  partially  bal¬ 
anced  incomplete  block  (PBIB)  design  with  b  (no.  of  blocks)  =  n  and  r  (no. 
of  replications)  =  k.  If  objects  i  and  j  are  a-th  associates  so  are  i  and 
n-j.  Thus  the  number  m  of  associate  classes  is  at  most  -|(n-l)  for  n  odd 
and  -|n  for  n  even,  but  may  be  less,  with  m  =  1  for  a  balanced  (BIB)  design. 
An  additional  feature  of  a  cyclic  set  is  that  each  object  occurs  once  in  each 
position  within  a  block.  Order  effects  are  therefore  automatically  bal¬ 
anced  out  and  the  sets  are  Youden  Type  designs,  balanced  (m  =1)  or 
partially  balanced  (m  >  1). 

The  same  procedure  can  be  used  for  any  n  and  k  except  that  when  n 
and  k  are  not  relative  primes  fractional  sets  arise  consisting  of  n/d 
blocks,  where  d  is  any  common  divisor  of  n  and  k.  In  terms  of  Fig.  1 
such  sets  correspond  to  arrangements  of  beads  which  can  be  reproduced 
in  fewer  than  n  rotations  of  the  necklace. 
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For  the  purpose  of  systematically  enumerating  all  cyclic  sets  it  is 
convenient  to  characterize  each  set  by  a  circular  partition  of  n.  Thus  we 
may  replace  {Ox^  x?.  .  .  x^_2  x^}  by  (x^  x^x^,  x3*x2’  '  '  '  ' 

’  "‘Vi*' 

Example  1,  For  n  =  8,  k  =  4  the  set  {0123}  becomes  (1115),  The 
cyclic  sets  may  now  be  written  down  in  increasing  order  of  the  numerical 
value  of  the  corresponding  partition:  (1115),  (1124),  (1133),  (1142),  (1214), 
(1223),  (1232),  (1313),  (1322),  (2222).  After  (1142)  we  omit  (1151)  this  being 
identical  with  (1115),  etc.  As  the  repetition  of  digits  indicates  the  set 
(1313)  consists  of  the  4  blocks 

0145  1256  •  2367  3470  (r  =  2) 

and  (2222)  of  the  2  (disconnected)  blocks  0246,  1357  (r  «=  1),  These  are 
still  PBIB  designs  but,  of  course,  no  longer  of  the  Youden  Type,  We 
shall  say  that  the  corresponding  arrangements  of  beads  on  a  necklace  have 
periods  4  and  2,  respectively.  As  a  check  note  that  all  (|)  blocks  are 
accounted  for  since  8x8+4+2  *  70, 

For  any  n  and  k,  the  total  number  of  sets,  being  equal  to  the  number 
of  distinct  arrangements  of  k  white  beads  and  n-k  black  beads  on  a  neck* 
lace  of  n  beads  (which  may  not  be  turned  over)  is  given  by  (Jablonskl  [8]  ) 

(2)  N(k,  n-k)  .1  Z  *  (d)  (kyd)(ydj|il.k)/J]  ,  ■ 


where  the  summation  is  over  all  integers  d  (including  unity)  which  are 
divisors  of  both  k  and  n-k,  and  b  (*)  is  Euler's  function,  the  number 
of  integers  less  than  and  prime  to  x.  Thus 


N(4,  4)  -i 


( 


8,' 

4!  4! 


+ 


4) 

2,'  2! 


+  2, 


2! 

1!  1! 


)  -  10. 


The  number  of  cyclic  sets  of  various  sizes  making  up  this  total  is 
tabulated  in  [7]  for  n  £  15. 
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If  a  design  of  size  n  =  b  =  7  and  k  =  r  =  3  is  required  a  look  at  the 
association  schemes  of  the  5  sets  in  (1)  leads  to  {013}  or  {015}  ,  both 
being  BIB  designs.  For  most  sizes  there  will  be  no  balanced  set  and  the 
choice  is  less  clear  but  might  be  based  on  the  usual  efficiency  factor. 
Combinations  of  sets  provide  larger  designs  and  again  the  question  of  opti¬ 
mal  selection  of  sets  arises.  This  presents  a  formidable  task  for  all  but 
small  designs.  Our  principal  aim  is  to  show  that  this  task  can  be  greatly 
simplified  if  certain  isomorphisms  between  cyclic  sets  are  recognised. 

A  systematic  approach  for  the  construction  of  optimal  cyclic  designs  is 
then  developed. 


3.  EQUIVALENCE  CLASSES.  Let  us  now  apply  to  {012}  of  equation  (1) 
the  re-numbering  or  permutation 


R(7,  3)  =(J 


1  2 
3  6 


3  4  5  6, 

2  5  14' 


obtained  by  multiplying  each  of  the  7  labels  by  3  (mod  7).  Then  {012} 
becomes 


036  362  625  251  514  140  403, 

a  Youden  Type  design  which  is  mersly  a  re -arrangement  of  {014}  ,  We 
write  {012} {.014}.  Thus  {.012}  end  (014}  are  isomorphic.  Two  further 
applications  of  R( 7 ,  3)  give  {024}  and  the  original  {012}  .  Ws  have 
therefore  established  the  equivalence  class  {012}  ~  {.014}  ■ —  {024}  .  No 
blocks  need  be  written  in  the  process  If  partition  notation  ia  used: 
{012}M036}  ■  (  331)  =  (133)  =  {014}^-{035}  «(322)  -  (223)  •  (024j  . 
Likewiae  [oil]*-  [032]  =  {023}  «  (214)  *  (142)  *  (ois)  ,  so  that  (013), 

[015]  form  a  second  equivalence  cites, 


The  same  procedure  can  be  used  for  any  prime  n  and  any  k,  To  see 
this  note  that  the  permutations  R(n,  1)  (the  identity  permutation), 

R(n,  2),  1  .  .  ,  R(n,  n-1),  form  a  group  under  "multiplication"  *  defined  by 


(2)  R(n,  i)  *  R(n,  j)  ■  R(n,  ij  mod  n) 


which  is  isomorphic  withl  he  multiplicative  group  of  residues  mod  n. 
Hence  all  dementi  ?(n,  i,  are  generated  by  powers  of  R(n,  g),  where 
gist  primitive  root  of  n  (i,  e .  ,  gx  ^  1  mod  n  for  x  «  1,  2,  n-2  but 
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gn  =1  mod  n).  But  a  permutation  <r  which  changes  one  cyclic  set  into 
another  must  be  of  the  form  R(n,  i)  if  we  assume  without  loss  of  gener¬ 
ality  that  cr  leaves  0  unchanged;  for  if  a,  b,  c,  d,  are  elements  of  the 
residue  set  with  a  and  b  =  a+d  two  elements  in  the  same  block  we  require 
that 


cr  (b)  -  cr  (a)  =  tr  (d)  all  a,  b,  d 

or  o'  (a)  +  cr  (d)  =  cr  (a+d), 

showing  that  cr  is  multiplicative:  cr  (a)  =  ca.  Thus  all  possible 
isomorphisms  between  cyclic  sets  can  be  established  conveniently  by 
repeated  application  of  R(n,  g). 

When  n  is  not  prime  the  R(n,  i)  continue  to  form  a  group  under  *  of 
(2)  provided  i  and  j  are  restricted  to  be  integer srelatively  prime  to  n.  The 
group  is  now  of  order  <f>(n)  and  is  clearly  isomorphic  with  the  multiplica¬ 
tive  group  of  the  reduced  set  of  residues,  g  is  said  to  be  a  primitive  root 
of  n  if  cj>(n)  is  the  smallest  power  making  g  $(  n)  ”  1  mod  n.  Primitive 
roots  exist  only  if  n  equals  2,  4,  pn,  or  2pn,  where  p  is  any  prime  >  2 
and  n  any  integer.  For  values  of  n  admitting  a  primitive  root  we  proceed 
as  before;  otherwise,  multiplication  by  each  member  of  the  reduced  set 
of  residues  will  establish  most  isomorphisms. 

Example  1  (cont’d,  )  Since  8  does  not  have  a  primitive  root  we  begin 
by  applying  R(8,  3)  to  the  sets  of  Example  1  and  find 

(1115)  —+  (1232)  ,  (1124)  ^-*(1223)  ,  (1142)  -^—(1322)  . 

The  other  sets  are  unchanged  by  the  transformation.  Likewise  R(8,  5) 
gives 

(1115)  ~  (1232)  ,  (1124)  ~(1322)  ,  (1142)  -^-*(1223)  . 


R(8,  7)  produces  "mirror  images"  obtained  by  reading  a  circular  partition 
anti-clockwise  rather  than  clockwise.  E.  g.  (1124)  -^—*(4211)  =  (1142).  This 
isomorphism  had  already  been  established  by  R(8,  3)  and  R(8.  5)  because 
5  -  -3  .  However,  an  additional  isomorphism  can  be  obtained  by  the 

permutation 


(°) 

'o' 


<}> 


<6  2> 


(3?) 

'7  y 


0  <5> 
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which  takes  (1133)  into  (1214).  This  is  the  only  instance  we  have  come  across 
where  the  equivalence  of  two  cyclic  sets  cannot  be  demonstrated  by  a  multi¬ 
plicative  permutation. 

A  listing  of  all  equivalence  classes  for  cyclic  sets  in  experiments  with 
n  <  15  and  k  =  3,  4,  5,  is  given  in  [12]  .  The  efficiences  of  these  sets 


regarded  as 

designs  have 

also  been  tabulated. 

When  n 

11 

00 

=  4  we  find 

Design 

E 

Ei 

E2 

E3 

E4 

{0123}  = 

(1115) 

.  812 

•  922 

•  8  34 

•  760 

•  712 

(01 24]  = 

(1124) 

.  851 

•  867 

•  87  3 

•  810 

•  868 

{0125]  = 

(1133) 

.  851 

•  867 

•  809 

•  867 

•  877 

{0134]  = 

(1214) 

•  836 

•  863 

.  810 

-  869 

•  807 

i0145}  = 

(1313) 

•  779 

*  802 

•  803 

•  668 

.  800  (r  =  2). 

Here  E  is  the  overall  efficiency  and  E  (j  -  1,  2,  3,  4)  is  the  efficiency 

factor  relating  to  the  comparison  of  j-th  associates.  On  the  basis  of  E  the 
choice  of  optimal  design  for  r  =  4  among  the  five  sets  (the  fifth  duplicated) 
lies  between  {.0124}  and  fj0125]  ,  with  the  latter  preferable  in  having 

only  3  associate  classes.  It  should  be  noted  that  except  for  fully  balanced 
designs  the  highest  value  of  E  does  not  necessarily  correspond  to  the  design 
with  the  smallest  number  of  associate  classes.  Other  optimality  criteria 
might  be  used  but  the  choice  of  cyclic  design  is  in  any  case  reduced  to  one 
of  the  non-isomorphic  sets.  Moreover,  it  is  only  combinations  of  these 
sets  (and  possible  disconnected  sets)  which  need  to  be  considered  in  the 
construction  of  larger  cyclic  designs.  In  Table  1  we  list  the  most  efficient 
cyclic  sets  for  n  <  15  and  k  =  3,  4,  5. 

Cyclic  sets  with  two  associate  classes.  For  purposes  of  comparison 
we  have  made  a  corresponding  compilation  in  Table  2  of  two-associate 
PBIB  designs  of  all  types  as  given  by  Bose  et  al.  [l]  and  (with  asterisks) 
by  Clatworthy  [4]  .  The  BIB  designs  in  this  range  are  also  included.  It 
will  be  noted  that  Table  2  has  gaps  for  several  (n,  k)  combinations 
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although  the  symmetrical  case  is  favorable  to  the  existence  of  designs  with 
a  high  degree  of  balance.  The  table  also  shows  that  a  cyclic  design  with 
more  than  t«n  associate  daises  m4y  ue  more  efficient  than  any  two-associate 


PBIB. 


It  is  of  some  interest  that  every  regular  (R)  group  divisible  PBIB  of 
Table  2  may  be  laid  out  as  a  cyclic  design;  this  is  already  done  in  [l]  in 
some  cases  and  may  be  effected  for  the  remaining  designs  by  suitable 
relabeling.  We  find  the  following  isomorphisms; 


n  =  6 

:  R1  ~ 

(013) 

R2 

(0124) 

n  =  8 

:  R  5  ^ 

(013) 

R.1 08*  — 

(01235} 

n  =  9 

:  R8 

(.0136)  , 

R112* 

(Ol  34  6} 

n  =  10 

:  R114*  ~ 

(01257)  ; 

n  a  12 

;  R15  -s. 

(0137)  , 

R116#  ^ 

(01356) 

RU7*  ~ 

(01249)  , 

Rllfi*  ^ 

(014710) 

n  *  14 

:  R24  ^ 

(0146}  ; 

n  ■  15 

R27  _ 

(0137) 

There  are  only  two  other  cyclic  designs  with  two  associate  classes  in  the 
range  under  consideration,  For  n  ■  13  we  have  Cl  —  014  ;  for  n  *12  the 
design  \ 01 2 4 ?}  has  the  same  association  scheme  aa  R116’*  but  is  not 
isomorphic  with  it, 

4.  COMBINATIONS  OF  CYCLIC  SETS,  Cyclic  sets  for  given  n  may 
be  combined  to  produce  a  wide  variety  of  cyclic  designs,  still  of  PBIB  form. 
This  can  always  be  done  if  the  number  of  replications  r  is  a  multiple  of  k 
but  will  also  be  possible  for  certain  other  values  of  r  if  fractional  seta 
exist.  We  shall  say  that  the  combined  design  is  of  aits  (n,  k,  r).  Equiva¬ 
lence  classes  may  again  be  established.  However,  the  most  efficient 
cyclic  design  of  given  size  is  not  necessarily  one  made  up  of  the  most 
efficient  cyclic  sets, 
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Example  2,  For  n  =  9>  k  =  3  we  have  the  equivalence  classes 
A  ;  (117)  ,  (225)  ,  (l44j  ; 

B  :  (126)  ,  (243)  ,  (153)  ,  (162)  ,  (234)  .  (135)  ; 

C  :  (333)  (r  *  1)  . 

The  order  within  a  class  has  been  arranged  so  that  successive  sets  are 
obtained  by  the  application  of  R(9,  2),  the  primitive  root  of  9  being  2. 
There  are  clearly  two  non-isomorphic  designs  of  siee  (9,  3,  4)  obtained 
by  combining  (333)  with  any  member  of  class  A  or  class  B,  Of  these 
the  latter,  which  may  be  written  as  (oi3,  036]  ,  is  the  more  efficient, 
with  E  ■  0,  713  and  4  associate  classes, 

To  get  designs  with  r  *  6  we  can  take  two  sets  from  A,  two  from  B, 
or  one  from  each.  Call  the  sets  A^,  Ag,  A^,  and  B^,  Bg,  ,  ,  ,  ,  B^,  We 

then  have  the  following  seven  equivalence  classes) 


A1A2 

i  A .  A .  i 

2  3 

’  A3A1  5 

BlV 

,  b2b3  , 

,  b3b4  , 

B4B5  •  B5B6  >  B6B1 

B1B3 

1  B2B4  1 

'  B3B3  ’ 

B4B6  '  B5Bl  *  B6B2 

B1B4 

,  b2b5 

'  B3B6  1 

AlV 

A2B2' 

A3B3 ; 

A1B4  ‘  A2B5  ’  A3B6 

A1B2 

•  A2B3 

-  A3B4  • 

A1  B5  '  A2B6  ’  A3  B1 

A1B3 

■  A2B4 

'  A3B5 

’  A1  B6  ’  A2B1'  A3B2 

Calculations  show  that  the  moat  efficient  cyclic  design  is  A,Ag  with 
E  ■  0,  7  31  and  4  associate  classes,  ‘  L 

The  present  example  has  been  chosen  to  bring  out  the  enumeration 
procedure  required  when  the  original  cyclic  sets  fall  into  several  equiva¬ 
lent.  j  classes, 
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Actually,  for  r  =  6  as  many  as  four  FBIB(2)  designs  are  available, 
viz,  SR13,  RIO,  LS3,  and  LS9*,  of  which  LS3  is  the  mm*  efficient  having 
£  =  0.  7*1 ,  When  r  =  4  the  only  tabulated  PBIB(2)  design  is  LS6,  with  the 
relatively  low  efficiency  E  =  0,  667,  For  r  10  Table  3  lists  a  selection 
of  cyclic  designs  in  cases  where  no  such  PBIB(2)  designs  are  known  to 
exist  or  are  all  of  more  than  trivially  inferior  efficiency. 

It  is  of  interest  to  note  that  the  number  of  non-isomorphic  designs 
made  up  of  s  sets  all  chosen  from  the  same  class  of  S  sets  is  just  N 
(s,  S-s),  where  N  is  defined  by  (2).  This  is  so  because  we  can  now  regard 
the  beads  of  Fig.  1  as  representing  sets  rather  than  blocks.  The  operation 
R(n,  g) .  where  g  is  a  primitive  root,  produces  a  unit  turn.  The  enumera¬ 
tion  of  non-isomorphic  designs  when  sets  are  from  more  than  one  class 
proceeds  exactly  as  described  in  (7]  for  k«  2. 

5.  FRACTIONAL  SETS.  The  number  nk  of  observations  required  for  a 
cyclic  set  of  size  (n,  k)  will  often  be  greater  thsui  desired,  especially 
whan  n  is  large,  In  this  situation  fractional  sets  are  very  useful.  As 
pointed  out  in  Example  1  such  sets  are  eharacteriasd  by  a  repetitive  pat¬ 
tern  in  their  partition  representation.  No  such  design  is  possible  if  n  is 
prime.  For  n  composite  fractional  sets  exist  corresponding  to  every 
divisor  d  (1  <  d  <  n)  of  n  since  there  must  be  at  least  one  partition  of  n 
consisting  of  d  repetitions.  Clearly,  k  must  be  a  multiple  of  d,  and 

r  *  k/d;  (however,  r  ■  1  gives  a  disconnected  set),  From  a  cyclic  set 
with  parameters  (n/d,  k/d)  a  fractional  set  with  parameters 
(n,  k,  r  =  k/d)  can  always  be  obtained. 

Example  3.  For  n  *  30  connected  fractional  sets  exist  for  k;»  4,  6, 

8,  9 1  10,  ,  ,  ,  .  Suppose  we  require  a  design  with  k  ■  6,  The  non-isomorphic 
connected  cyclic  sets  of  sise  (15,  3)  are  (1113),  (1212),  (1311),  (1410)  ,  and 
(159).  Of  these  (1212)  leads  to  the  most  efficient  design  of  sizeT30,  6,  3), 
vis,  (12121212)  or  (ol315_  16  18}  with  E  ■  0.  762 

In  [12]  a  selection  of  the  most  efficient  fractional  sets  of  given  sice 
is  tabulated  for  n  5100. 
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Table  1.  Most  efficient  symmetric  cyclic  PB1B  design  D  for  n  treatments 
and  block  size  k,  and  its  efficiency  E, 


n 

k=3 

D 

E 

k=4 

D 

E 

k=5 

D 

E 

6 

M  2 

'  784 

(0123) 

•  895 

(01234}  1 

■  961 

7 

f013)  1 

•778 

{0124}  1 

•  876 

(01234} 

•  932 

8 

(on)  2 

•  748 

1 0 1 2  5} 

•  851 

{01235}  2 

•  914 

9 

(on) 

■  722 

{0134} 

.  836 

(012  35} 

>  898 

10 

{on} 

•  700 

(0125) 

•  823 

(01245} 

.  886 

11 

(on) 

•  676  ' 

(0125} 

•  817 

(61247)  1 

.880 

12 

(014) 

'  673 

(0137}  2 

•  813 

(01247}  2 

•  870  * 

13 

(014)  2 

■  667 

{0139}  1 

■  812 

{01269} 

.  863 

14 

(014} 

•  670 

{0146}  2 

■  805 

{oi 358} 

•  859 

15 

(015) 

•  641 

[0137}  2 

•  795 

{pl2410) 

'  853 

N,  B.  Superscripts  2  denote  respectively  BIB  and  PBIB(2)  designs. 


Table  2  (BIB)  ar.  —  ^vc “ ii scciitc  p® dcsi j" 

efficiencies  from  Bose  at  &1.  [1]  and  Clatworthy*  [4]  , 


RIOS'*,  R109' 
LS10,  R112* 
RU4* 


R116* ,  R117* 


29? 


Table  3,  Selected  cyclic  deeigne  with  r>k,  corresponding  optimal  two- 
aaiociate  PBIB  deaigna,  and  efficiencies  E, 


Site 

(n,  k,  r) 

Cyclic 

design 

E 

PBIB(2} 

design 

8,  3,  6 

f  01 3 ,  014) 

•  756 

R50* 

•747 

8,  4,  5 

(oi34,  0246} 

■  850 

- 

9,  3,  4, 

foi3,  036) 

•  713 

LS6 

■  667 

10,  4,  6 

(0147,  0156} 

■  825 

T  3 

■789 

10,  4,  a 

(0126,  0148] 

'  830 

3U4 

823 

11,  3,  6 

(oi 3 ,  026) 

•  727 

■ 

11,  3,  9 

(oi3,  014,  027) 

•  730 

- 

U,  4,  8 

(0134,  0248} 

•  823 

- 

13,  4,  8 

(0125,  0159) 

*  807 

C2 

<  797 

13,  5,  10 

(01247,  01258} 

'  865 

■ 

14,  3,  9 

(014,  0211,  019} 

•  709 

• 

14,  5,  10 

(012410,  01710112) 

•  862 

m 

15,  3,  4 

(oi  5 ,  0310) 

•  682 

T23 

■  673 

15,  5,  6 

(01257,  036912) 

•  856 

T  38* 

'803 

SOME  RESULTS  ON  THE  FOUNDATIONS 
OF  STATISTICAL  DECISION  THEORY 


Bernard  Harris,  J.  D.  Church,  and  F .  V.  Atkinson 
Mathematics  Research  Center,  U.  S.  Army 
The  University  of  Wisconsin 

INTRODUCTION.  A  fundamental  problem  in  statistical  decision  theory 
is  concerned  with  establishing  criteria  for  selecting  a  single  decision 
procedure  from  the  set  of  available  decision  procedures.  In  this  paper, 
some  criteria  for  optimality  of  statistical  decision  procedures  are  proposed 
and  the  consequences  of  these  criteria  are  discussed.  It  is  shown  that 
these  optimality  criteria  exclude  a  very  general  class  of  decision  criteria, 
which  contain  as  members,  the  minimax  and  minimax  regret  criteria. 
Finally,  we  note  that  these  optimality  conditions  are  consistent,  in  that 
there  exists  a  decision  procedure  which  satisfies  all  conditions,  and  a  con¬ 
structive  procedure  is  given  for  determining  such  a  decision  procedure. 

THE  GENERAL  STATISTICAL  DECISION  PROBLEM.  A  statistical 
decision  problem  is  characterized  by  a  set  of  states  of  nature  S,  whose 
elements  will  be  denoted  by  s,  and  a  set  of  pure  (non- randomized)  deci¬ 
sions  D,  whose  elements  will  be  denoted  by  d.  The  statistician  selects 
an  element  d  from  D,  and  if  nature  is  in  state  s,  a  loss  L(d,  s)  is 
incurred.  An  experiment  is  conducted  and  random  variables  X^X^-,  .  .  .  ,  X^. 

are  observed  where  X^.X^,  .  .  .  ,  X^  has  the  probability  distribution 

P(X1’X2’  ‘  '  ’  ’  x]sj  I  s)*  recluire  that  the  distributions  P^.x^ . |  s) 

be  distinct  for  every  s  e  S.  Then,  since  the  decision  is  to  be  made  follow¬ 
ing  the  experiment,  the  decision  procedure  is  a  function  6  from  the  sample 
space  to  the  space  of  decisions  D.  Let  A  be  the  set  of  such  functions  and 
note  that  d  is  then  a  random  variable,  i.  e.  d=  SfX^jX^,  .  .  .  >X^).  This 
risk  function  p(  5  ,  s)  is  then  defined  by 

E[L(d,  s)]  =  p(5  ,  s). 

The  statistician's  objective  is  to  choose  6  ,  so  that  p(6  ,  s)  is  small  in  some 
appropriate  sense.  It  will  frequently  be  desirable  (in  the  sense  of  reducing 
risk)  to  augment  the  set  of  decisions  to  include  the  randomized  decisions; 
and  equivalently  to  augment  the  set  of  decision  procedures  A  to  J  the  set 
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of  randomized  decision  procedures,  whose  elements  will  be  denoted  by 
5.  is  the  set  of  all  probability  mixtures  of  elements  o:  a. 

The  fundamental  problem  of  statistical  decision  theory  is  to  decide 
how  to  choose  an  element  $  ■  £•  We  can  interpret  this  as  consisting  of 
two  sub-problems. 

1.  What  conditions  should  be  imposed  on  a  randomized  strategy  ♦  « 

so  that  we  can  regard  strategies  having  thoss  properties  as  being  optimal? 

2,  Having  decided  which  conditions  are  appropriate,  how  do  we  deter¬ 
mine  which  elements  $  t  <£  satisfy  those  conditions  ?  Note  that  for  some 
sets  of  possible  conditions  which  one  may  wish  to  consider,  it  may  happen 
that  there  are  no  strategies  in  {  which  satisfy  them. 

We  will  make  the  formal  assumption  that,  in  advance  of  the  experiment, 
the  statistician  is  in  "complete  ignorance"  of  which  element  s  of  S  has 
been  selected  by  nature.  That  is,  that  there  is  no  a  priori  information 
available  concerning  the  mechanism  by  which  nature  Will  select  an  element 
s  1  S. 

The  results  stated  in  the  succeeding  sections  have  been  setabliehed 
under  the  following  hypotheses. 

1.  S  and  D  are  finite  eets,  i.  e.  S  ■  (e^,  *2'  ‘  •  •  1  •n),  D  ■  d2 . 

2.  With  probability  one ,  the  random  vector  (Xj.X^,  ..  .  ,X^)  assumes 
only  a  finite  number  of  values. 

As  a  consequence  of  the  two  hypotheses  stated  above,  A  is  a  finite  set, 

and  we  can  label  its  elements  as  5,,  6, . 6 

1  £  m 

Despite  the  restrictive  nature  of  these  assumptions,  thsre  ars  a 
substantial  number  of  statistical  problems  to  which  they  are  applicable, 
and  in  addition,  many  problems  may  be  approximate  by  problems  satisfy¬ 
ing  the  above  hypotheses.  As  an  example  of  a  problem  which  satisfies  the 
above  restrictions,  consider  the  following  illustration. 

Let  X^.X^, .  .  .  ,XV.  be  independent  and  identically  distributed  random 
variable!  with 
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Ptxi=i]  =pr  p(x(  =  0}  - 1  -  P).  0<p,  <1 

for  i  e  l,  2, ,  .  .  ,Nj  j  a  i,  2;  and  S  =  (l,  2}  ,  Then,  the  sample  space  has  2n 
elements,  If  we  let  D  =  (l,2j  ,  then  A  consists  of  all  functions  from  the 

sample  space  to  D,  and  hence  A  has  2^  elements,  Heacs,  for  this  prob¬ 
lem  the  above  assumptions  are  all  satisfied  . 

We  can  make  this  illustration  more  concrete  by  noting  that  the  above 
is  essentially  the  problem  of  testing  whether  a  coin  is  fair  (p  *  i.)  or  has 

3  i  Ct 

probability  (p^  =  — )  of  landing  heads,  We  can  interpret  the  two  elements  of 

D  as  being  1:  Accept  the  hypothesis  that  i  *  1,  i.e,  p  *  j  s  2:  Accept  the 

hypothesis  that  s  ■  2,  i,  e.  p  ■  —  ,  Thus,  the  illustration  given  Is  an 

"abstraction"  of  a  test  of  a  simple  hypothesis  against  a  simple  Slternativ* 
in  a  coin  tossing  problem. 

It  is  well-known,  that  as  a  consequence  of  the  above  two  assumptions 
we  can  Identify  the  selection  of  a  dacision  procedure  $  with  the  saleotioh^ 
of  a  point  in  a  convex  polyhedron  C  in  Euclidean  n-epace,  where  C  li 
generated  as  the  convex  hull  of  the  points  (p( p(fi1«2),  , ,  .  , 

p(filBn))>  5  «  A.  If  we  define  the  matrix  A,  whose  element*  are  a^. 

i  ■  1, 2,  .  ,  .  ,m;  J  ■  1,  2 . n  by  p( 6 ^  b  ,  then  C  ■  C(A),  the  convex 

hull  of  the  row  vector*  of  A,  Thus,  we  can  use  the  natural  relationship 
between  the  matrix  A  and  the  polyhedron  C(A),  and  freely  characterise 
all  relevant  aspects  of  the  problem  in  term*  of  either  the  matrix  or  the 
associated  polyhedron,  The  reader  ie  referred  to  the  book  by  D,  Blackwell 
and  M.  A.  Girshick  [l]  for  the  relevant  details, 

W#  now  turn  to  the  characterisation  of  desirable  properties  for 
decision  procedures. 

THE  CHOICE  OF  A  DECISION  PROCEDURE,  It  is  convenient  at  this  time 
to  introduce  eome  definitions  which  will  be  needed  in  order  tc  specify  thoee 
properties  of  a  decision  procedure  which  will  be  considered  desirable, 
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Definition  1,  Two  decision  procedures  b^,  b2’  in  3>  will  be  said  to  be 
equivalent  if 

p(b^  »j)  ■  P(4>2 .  Ij)  for  j  ■  1,  2 . n 

Definition  2,  is  said  to  be  dominated  by  b2  if 

p(b2-  «j)  S  p(bj,  «j).  j  ■  1.  2, .  .  .  ;  ft 

with  strict  inequa'ity  holding  for  at  least  one  j. 

Note  that  if  bj  is  dominated  by  bj>  then  regardleii  of  which  state 
of  nature  s^  has  been  selected  by  nature,  the  risk  using  bj  is  always 
at  least  as  large  as  that  using  bj>  and  hsncs  b2  is  always  to  bs  preferred, 
over  b^ 

Definition  3.  A  decision  procedure  <s  ^  is  said  to  bs  admissible  if  it- is  not 
dominated  ty  any  element  0  <  5  • 

Since  we  have  previously  noted  that  dominated  strategies  are  not 
desirable,  then  clearly  the  selection  of  a  strategy  should  bs  mads  from 
among  those  that  are  admissible. 

Definition  4.  A  decision  procedure  d-  is  essential  if  it  is  admissible  and 
if  for  every  pair  of  decision  procedures  ,  with  4»j  not  equivalent 

to  $q,  and  for  every  real  number  X,  0  <  X  <1, 

p(b0. 8j )  i  xp(b1(  Sj)  +  (ux)p(b2,  Sj) 

for  at  least  ono  index  J,  1  Sj  Sn. 

The  essential  decision  procedures  are  those  which  are  admissible 
and  in  addition  are  also  extreme  points  of  ths  convex  polyhedron  C(A), 
These  decisions  can  then  be  used  to  generate  all  strategies  which  one  may 
wish  to  consider. 
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where  X  is  a  positive  reel  number  end  the  vector  (c, ,  c, , . . . c  )  is  en 

12  n 

arbitrary  real  vector,  then 

Q(Al)  ■  {Xx+7  ,  H  i  Q(A0)  ,  «  (c1,  c2,  .  ,  ,  ,cn)}  . 


This  requirement,  includes,  for  example,  invariance  under  the 
change  of  units  of  the  loss  function,  In  particular,  if  X  ■  1  and  C.  ■ 

J 

-min  p(6  ,  s),  the  matrix  A.  is  reduced  to  its  regret  matrix  , 
1  S  i  SL  m  1  j 


6,  If  C(AjT)  ■  C(A2T),  where  A^  is  the  transpose  of  A,  and  in 
addition  A^  can  be  obtained  from  A2  by  deleting  j  columns  from  Ag, 
then  Q(A^)  can  be  obtained  by  deleting  the  corresponding  coordinates  from 
every  vector  in  Q(A2), 


Property  6  Includes  the  column  duplication  property  required  by  other 
writers,  such  as  J,  Mllnor  [4]  .  The  point  of  this  property,  is  that  under 
complete  ignorance,  the  decision  problem  for  the  statistician  is  essentially 
the  same  in  both  cases, 


7.  Lst  E^  be  the  submatrix  of  A  corresponding  to  essential  decision 
procedures  in  A,  Then,  if  Aj  and  A 2  are  two  matrices  with 

C(Ea  )  •  C(Ea  ),  we  require  that  Q(Aj)  ■  Q(Aj)  , 


This  says  that  the  set  of  optimal  decision  procedures  should  depend 
only  on  thoes  pure  strategies  which  are  candidates  for  good  strategies, 

We  might  note  that  a  risk  vector  'x'  «  C(A)  is  an  essential  strategy  if  and 
only  if  it  uniquely  minimises  the  risk  for  some  a  priori  distribution  on  the 
states  of  nature. 

8,  If  {a.]  is  a  sequence  of  matrices  with  lim  A  "A,,,  and 
^  J  -*  «• 

Xji  Q(Aj)  for  svsry  j  2  1,  then  svsry  limit  point  of  (x^  is  an  element  of 

Q(A0). 
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This  last  condition  is  &  continuity  requirement  The  resdcr'r  intuition 
may  be  aided  by  noting,  that  if  one  statistical  decision  problem  may  be 
approximated  by  another  statistical  dscision  problem,  then  this  property 
requires  that  optimal  decision  procedures  for  the  first  problem  are  also 
approximated  by  the  optimal  decision  procadures  for  the  second  problem. 

R.  D.  Luce  and  H,  Raiffa  [3]  give  an  extensive  diecueeion  of  similar 
ayateme  of  optimal  properties.  The  reader's  attention  is  also  specifically 
directed  to  papere  by  H,  Chernoff  [2]  and  J,  Milnor  [4]  ,  which  deal  with 
thie  problem, 

CONSEQUENCES  OF  THIS  CHOICE  OF  DESIRABLE  PROPERTIES. 

Let  v  ■  min  p{  6  ,  a  )  and  define  “v  •  fv, ,  v  v  ).  befine 

J  liiim  1  J  1  2  n 

L 

|  Cxi1  ■  (E  |  x.  I  p)p  ,  1  spi«,  where  I  Cxi  I  ie  interpreted  as 
1  1  P  i»l  1  • 

•up  |  x  |  .  Then,  let  the  class  of  dscieion  procedures  A  (1  jp 
lSiSn  P 

specify  as  optimal  all  "x  <  C(A)  which  are  admissible  and  satisfy 


l!~-'*llp  s  I!~-"y||p 

for  all  y  <  C(A).  Than,  the  following  theorem  can  be  established, 

THEOREM,  For  1  $p  <«,  satisfies  every  property  with  the 
exception  of  property  6,  A  eatisfies  every  property  except  property  8. 

The  reader  should  note  that  A^  is  Laplace's  criterion  and  that  A^ 

is  the  minimax-regret  criterion  restricted  to  admissible  decision  proce¬ 
dural.  The  failure  of  the  minimax  regret  criterion  to  satiafy  the  above 
Hat  of  properties  also  establishes  that  the  minimax  criterion  does  not 
always  satisfy  the  list  of  requirements  given  above. 

Finally,  we  have  the  following  theorem,  ) 
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THEUKEM,  There  ia  at  leaat  one  deciaion  procedure  satisfying  all  of 
the  above  propertlea, 

The  proof  of  thia  laat  atatement  ia  accompliahed  by  exhibiting  a  con¬ 
structive  proceaa,  which  we  now  sketch. 


Let  ,  j  =  1,  2, , . .  be  a  monotone  non-increaaing  aequence  of 

positive  real  numbera,  with  lim  «  =  0.  Let  dfjT.'y)  =  aup  |  x  -y  |  and 

j-»  J  1  < i <  n 

let  Q  =  C(A),  Define  =  min  x.  and  ~  . v^),  Then 

x  i 

let  i.  =  min  dfvjfx).  We  now  proceed  inductively.  For  hg  1,  define 
1  x  <  Q  1 

<-*-•  (h) 

Qh+1  =  ('"x  «  •  dCv^.Tt)  £  where  v^  ;  a^rnin  x  and 

-3  .1  ,3  1  'x'  I  a. 


v  »  (v|h\  .  .  ,  ,  v^)  and  a.  =  min  d(v!  ,'S).  Then,  it  can  be  ahown 

h  l  2  n  h  . — -  _  h 

x,  oh 


that  Q(A)  a  A  Q.  aatiafiea  all  of  the  requirements. 
h-1  h 

One  of  the  consequences  of  the  above  conatruction  ia  that  Q(A)  ia  a 
single  point  T,  However,  the  specific  single  point  obtained  may  depend 
on  the  choice  of  the  sequence  employed, 

The  reader's  intuition  concerning  the  above  construction  may  be 
aided  by  considering  the  process  as  a  limit  of  a  sequence  of  minimax- 
regret  procedures,  as  follows; 

ia  the  minimax  regret  decision  procedure  (more  properly,  it  is 

the  distance  of  the  risk  vector  associated  with  the  minimax  regret  decision 
procedure  from  /v'  =  'vj).  Then  a  new  convex  polyhedron  ,  ia  constructed, 
containing  the  risk  vector  for  the  minimax  regret  decision  procedure ,  and 
the  minimax  regret  decision  procedure  for  is  determined,  The  process 
is  repeated  and  converges  to  a  single  point  a  =  Q(A), 
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PATHOPHYSIOLOGY  OF  INDIAN  COBRA  VENOM 


A.  Vick,  Henry  P  CUuehta.  and  Jamee  H.  Manthei 
Directorate  of  Medical  Research 
US  Army  Edgewood  Arsenal 
Chemical  Reeeareh  and  Development  Laboratories 
Edgewood  Arsenal,  Maryland 


INTRODUCTION,  It  has  been  reported  that  the  venom  of  the  hooded 
cobra,  Naja  naja,  has  a  detrimental  effect  on  the  respiratory  system  of 
animals  and  man  [1-3],  Several  workers  have  attempted  to  fractionate  the 
crude  venom  into  its  various  toxic  fractions  [4,5]  ,  they  being;  (a) 
neurotoxic  fraction,  (b)  a  cardiotoxic  fraction  and  (c)  a  non-specific 
hemolytic  fraction.  Our  study  is  concerned  with  ths  sffect  of  crude  cobra 
venom  on  cortical  electrical  activity  (EEG),  reipiration  and  the  cardio- 
vaacular  system. 

MATERIALS  AND  METHODS,  In  this  etudy  a  total  of  54  dogs  and 
5  monkeys  of  the  Cynapthecoid  group  (sooty  mangabey)  were  used,  Of 
the  above  total,  44  adult  mongrel  doge,  anesthetized  with  sodium  pento¬ 
barbital,  30  mg/kg,  were  uaed  to  study  the  effect  of  venom  on  the 
respiratory  and  cardiovascular  ayatsms,  Fsmoral  arterial  pressure 
waa  monitored  using  a  Statham  strain  gauge  and  a  Grass  polygraph 
recorder.  The  phrenic  nerve  waa  isolated  at  the  level  of  ths  5th 
cervical  vertebra.  The  nerve  was  carefully  dissected  free  of  connective 
tissue  and  sectioned.  Silver  wire  electrodes  were  connected  to  the  cen¬ 
tral  end  of  the  phrenic  from  which  nerve  lmpuleee  were  then  monitored 
and  amplified  on  a  Tektronix  oecilloecope.  Permanent  recordings  were 
obtained  photographically.  Both  EKG  and  reepiratory  rate  were 
recorded  in  eome  of  the  animals  ueing  a  Graai  polygraph  recorder,  All 
of  the  above  animals  were  administered  (0.  5  mg/kg)  lyophillsed  crude 
cobra  venom,  which  was  reconstituted  with  normal  saline  and  injected 
directly  into  the  femoral  vein, 

The  44  animale  were  divided  into  the  following  groups:  Group  I 
was  comprised  of  six  animale  used  to  study  the  overall  effect  of  the  venom 
on  blood  pressure  and  respiration.  Group  II  •  Eight  animale  ventilated 
with  a  Starling  pump  at  respiratory  arrest  but  prior  to  cardiovascular 
failure.  The  resultant  effect  on  survival  time  was  noted.  Group  III  > 

The  remaining  thirty  animale  were  used  to  study  specific  effect*  of  the 
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ot  the  venom  on  tne  respiratory  syetem.  including  the  phrenic  nerve 
and  diaphragm,  Nerve  impulses  over  the  central  end  of  the  cut  phrenic 
nerve  were  continuously  observed.  The  peripheral  end  of  the  cut  phrenic 
nerve  and  the  diaphragm  were  etimulated  at  intervals  using  a  Grass  model 
34  stimulator.  Diaphragmatic  muscle  contractions  ware  recorded  with 
a  Grass  Force  Displacement  Transducer,  The  effect  of  venom,  artificial 
respiration  and  changes  in  pO.,  and  pCO,  tension  on  nerve  activity  were 

observed.  Group  IV  comprised  the  remaining  ten  doge  and  five  monkeys 
which  were  ueed  to  monitor  the  effect  of  crude  cobra  venom  (0,  5  mg/kg) 
on  cortical  electrical  activity,  Blood  preesure  and  respiratory  effects 
were  also  recorded.  The  cortical  electrical  activity  was  recorded  using 
bipolar  ellver  electrodes  which  were  surgically  implanted  directly  on  the 
dura  of  each  hemisphere  of  the  brain.  Continuous  electroencephalograms 
were  recorded  prior  to  and  for  up  to  10  hours  after  the  intravenous  ad- 
minstration  of  the  venom. 

RESULTS. 

Group  I.  The  effect  of  venom  on  respiratory  rate  and  arterial  blood 
pressure  is  shown  In  Figure  1.  Within  1-5  minutes  post-lnjaetlon  th»ri 
is  an  increase  in  respiratory  rats  as  wall  as  a  sharp  drop  in  blood  pres¬ 
sure.  This  is  followed  by  a  progressive  decrease  in  respiratory  rate 
and  volume  to  complete  arrest  at  90-120  minutes,  During  this  time  blood 
pressure  makes  a  partial  recovery  remaining  stable  until  respiratory 
failure,  at  which  time  cardiovascular  collapse  results,  The  average 
survival  time  of  this  group  was  105  minutes, 

Group  II,  The  effect  of  venom  on  the  artlflcally  ventilated  animal 
is  shown  In  Figure  2,  These  animals  were  placed  on  a  positive  pressure 
respirator  at  time  of  respiratory  cessation,  with  a  resultant  increase  in 
survival  time  of  from  4-6  hours,  However,  all  animals  untimately 
developed  arrhythmias  and  progrsssivs  hypotension  which  led  to  death 
Figure  3.  The  average  survival  time  for  this  group  of  animals  was  7,  5 
hours  post-venom, 

Group  III,  Changes  in  phrenic  nerve  action  potentials  Induced  by 
cobra  venom  are  shown  in  Figure  4,  Action  potentials  prior  to  venom 
are  synchronous  corresponding  to  the  inspiratory  phase  of  respiration, 
Increase  in  both  rste  and  amplitude  are  noted  within  1-5  minutes  aftsr 
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administration  of  venom.  The  central  component  of  the  nerve  continues 
to  discharge  for  from  5-10  minutes  after  complete  cessation  of  respira¬ 
tion.  During  this  period  phasic  discharges  over  the  phrenic  nerve  become 
sporadic  and  irregular.  These  central  impulses  are  eliminated  by  placing 
the  animal  on  the  artificial  respirator.  At  any  time  prior  to  death  impulse 
traffic  can  again  be  re-established  by  discontinuing  artificial  respiration, 
even  though  the  animals  do  not  breathe  spontaneously.  Phrenic  impulses, 
as  seen  on  the  oscilloscope,  continue  with  increasing  frequency  and 
amplitude  until  the  animal  either  expires  or  is  again  ventilated. 

The  administration  of  5  percent  CO^  to  artifically  ventilated  animals 

initiates  discharges  over  the  phrenic  nerve.  This  is  quickly  eliminated  by 
removal  of  the  stimulus.  Phasic  phrenic  discharges  can  also  be  elicited 
in  ventilated  animals  by  the  reduction  of  their  tidal  volume.  Where  such 
activity  is  noted  the  administration  of  100  percent  oxygen  does  not 
eliminate  or  appreciably  alter  their  frequency  or  amplitude. 

The  terminal  effect  of  venom  on  impulse  traffic  over  the  phrenic 
nerve  is  characterized  by  abnormal  appearing  bursts  probably  due  to  a 
combination  of  hypotension  and  central  nervous  system  ischemia. 

Spontaneous  contractions  of  the  diaphragm  show  a  gradual  decrease 
in  force  of  contraction  after  venom  ultimately  leading  to  complete 
cessation  of  movement  Figure  5  [6]  . 

Group  IV.  The  effect  of  crude  cobra  venom  (0.  5  mg/kg)  on  the  EEG 
of  the  dog  and  monkey  can  be  seen  in  Figure  6.  Within  30-60  seconds 
following  the  administration  of  the  venom  there  was  complete  loss  of 
EEG,  as  well  as  corneal  reflexes.  There  also  occurred  a  sharp  drop 
in  arterial  blood  pressure  shortly  after  cessation  of  all  EEG  activity. 

This  hypotension  was  followed  by  a  partial  recovery.  The  effect  of  the 
venom  on  EEG  was  irreversible.  As  seen  in  Table  I  all  animals  expired, 
with  an  average  survival  time  of  1.  4  hours  in  the  dog  and  2.  0  hours  in  the 
monkey. 

DISCUSSION.  This  study  has  characterized  the  effects  of  crude 
cobra  venom  (0.  5  mg/kg)  on  the  peripheral  respiratory  mechanism, 
cardiovascular  system  and  cortical  electrical  activity  (EEG)  of  the  dog 
and  monkey.  The  respiratory  effect  is  apparently  due  to  a  blockage  of 
nerve  impulses  at  the  neuromuscular  junction  of  the  diaphragm.  This 
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is  supported  by  the  fact  that  the  respiratory  center  remains  functional  after 
venom.  There  are  continued  phrenic  dischagres,  although  somewhat 
modified  following  the  venom.  The  muscle  of  the  diaphragm  remains  in 
tact  in  that  it  retains  its  response  to  stimuli.  This  same  stimulation  when 
applied  to  the  phrenic  nerve  produces  no  response  in  the  diaphragm.  It 
appears,  therefore,  that  transmission  of  impulses  is  interferred  with  at 
the  level  of  the  neuromuscular  junction.  The  character  of  this  block  is 
unknown. 

The  primary  lethal  effect  of  cobra  venom,  respiratory  arrest,  was 
shown  to  be  alleviated  with  the  application  of  artificial  ventilation.  This, 
however,  was  a  temporary  phenomena  in  that  all  animals  eventually 
developed  cardiovascular  failure.  The  etiology  of  this  phenomenon  has 
not  been  studied  but  may  be  related  to  the  action  of  venom  on  motor  end 
plates  [7]  .  The  effect  of  venom  on  cardiovascular  hemodynamics  may 
also  be  due  in  part  to  its  strong  hemolytic  effect,  producing  a  high  serum 
potassium  which  may  result  in  cardiac  failure  [6]  . 

The  cortical  electrical  activity  of  the  brain  of  the  dog  and  monkey  has 
been  shown  to  be  severely  depressed  by  the  action  of  cobra  venom.  The 
exact  action  of  venom  is  not  clear  but  may  also,  m  some  way,  be  related 
to  its  blocking  effect  on  neuromuscular  transmission  [8]  . 

SUMMARY.  This  study  has  dealt  with  the  effects  of  cobra  venom, 

Naja  naja,  on  the  respiratory  system  cardiovascular  system  and  the 
cortical  electrical  activity  of  the  dog  and  monkey.  Results  have  indicated 
that  death  is  primarily  due  to  respiratory  failure,  which  appears  due  to 
peripheral  neuromuscular  blockade.  The  character  of  this  block  is 
unknown.  The  respiratory  center,  phrenic  nerve  and  diaphragmatic  mus¬ 
cle  fibers  appear  to  be  relatively  unaffected  by  the  venom.  Survival  time 
was  increased  several  hours  with  artificial  ventilation,  however,  all 
eventually  developed  cardiovascular  difficulties  terminating  in  death. 

This  effect  may  be  due  to  the  extended  action  of  venom  on  the  areas  of 
the  body. 

In  addition,  venom  has  been  shown  to  have  a  severe  depressant  effect 
on  the  cortical  electrical  activity  of  the  dog  and  monkey.  The  exact 
mechanism  by  which  this  effect  is  produced  has  not  as  yet  been  defined. 


Design  of  Experiment* 


313 


REFERENCES 

1.  Ganguly,  S.N,  and  Malkana,  M.T.:  Indian  F.  med,  Rea.  24;  281,  1936. 

2.  Sarkar,  B,  B.  ,  Mitra,  S,  R  and  G  ho  ah,  B.N.:  Indian  F.  med.  Rea. 

30:453,  1942,  '  "  ~ 

3.  Gautrelet,  J,  ,  Halpern,  N,  and  Cortiggiani,  E , ;  Arch,  int,  Phyaiol. 

38:  293,  1934.  “  " 

4.  Maater,  R.W.P.  and  Rao,  S.S.;  F.biol.  Chem.  236:1986,  1961. 

5.  Sarkar,  N.  K, :  Ann,  Blochem.  exper,  Med,  Si  11 ,  1948. 

6.  Vick,  J .  A,  Ciuchta,  H.P,  and  Folley,  E.H.:  Arch,  int,  Pharma- 

codyn.  Vol.  153,  No.  2:424-429,  1965. 

7.  Chang,  C.C.  and  Lee,  C.Y.:  Arch,  int.  Pharmacodyn.  144:214,1963. 

8.  Vick,  J.  A.  ,  Ciuchta,  H.P.  and  Polley,  E.H.:  Nature,  Vol.  203, 

No.  4952:1387-1388,  Sept.  26,  1964. 


LEGENDS  FOR  ILLUSTRATIONS 

Figure  1,  The  effect  of  cobra  venom  on  arterial  blood  preiaure  and 
reapiratory  rate. 

Figure  2.  Modification  of  venom  effect  by  use  of  artificial  reepirator, 

Figure  3.  The  effect  of  cobra  venom  on  cardiovascular  function  after 
reapiratory  arreat  and  subsequent  artiflcal  ventilation. 

Figure  4.  Change «  in  phasic  phrenic  discharges  produced  by  cobra 
venom.  Effects  of  artificial  respiration  and  administra¬ 
tion  of  5  percent  CO^  after  cessation  o l  spontaneous  respira¬ 
tion  are  shown, 

Figure  5.  Effect  of  cobra  venom  on  blood  pressure,  phrenic  nerve 
discharges  and  diaphragmatic  contractions.  Note:  Lose 
of  diaphragmatic  response  to  direct  phrenic  stimulation  (PS), 
Diaphragmatic  responses  to  direct  stimulation  (DS)  are 
retained, 

Figure  6.  The  effect  of  cobra  venom  on  EEG  and  blood  pressure.  , 
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COMPUTER  ANALYSIS  OF  VISUAL  DISCRIMINATION  DATA 


JohnC.  Atkinson 
Directorate  of  Medical  Research, 
Chemical  Research  b  Development  Laboratories 
Edgewood  Arsenal,  Maryland 


One  of  the  methods  used  by  the  Directorate  of  Medical  Reeearch,  Chem¬ 
ical  Research  and  Development  Laboratories  in  evaluating  the  effect  of 
various  drugs  on  an  animal's  performance  is  a  visual  discrimination  test. 
This  is  a  conditioned  visual  discrimination  between  a  triangle  and  a  square 
in  which  monkeys  are  trained  to  avoid  or  escape  an  electric  shock  by 
pressing  a  lever  under  the  correct  symbol,  the  triangle.  Thus,  success¬ 
ful  performance  involves  sensory  perception  (vision)  decision  making  and 
motor  activity  (pressing  the  lever). 

If  a  drug  interfere  with  any  of  these  activities  the  result  will  be.  a 
slowed  or  inaccurate  performance.  An  obvious  correlation  can  be  seen 
between  this  test  and  many  tasks  performed  by  a  soldier  during  combat. 

In  our  operation  Rhesus  monkeys  are  used  as  test  subjects.  Each 
monkey  is  placed  in  a  sound  attenuated  booth  which  it  enclosed  to  prevent 
visual  as  wall  as  audio  distraction.  The  monkey  is  restrained  by  a  Wilinski 
harneis*[l]  ,  By  this  means  the  monkey  it  kept  in  front  of  a  panel  on  which 
there  ire  two  screens  at  an  equal  level.  At  the  beginning  of  a  trial  a 
triangle  appears  on  one  screen  and  a  square  on  the  other.  If  the  monkey 
presses  the  lever  under  the  triangle,  the  symbols  disappear  from  the 
screen  and  the  trial  is  over.  If  he  presses  the  lever  under  the  square  he 
receives  a  punishment  in  the  form  of  a  mild  electrical  shock  for  twenty 
(20)  milliseconds,  This  is  called  an  incorrect  response.  If  the  monkey 
does  not  press  the  designated  lever  in  an  interval  of  five  (5)  seconds  he 
receives  a  negative  reinforcement  in  the  form  of  a  mild  electrical  shock. 
This  shock  continues  for  five  (5)  seconds  unless  sooner  shut  off  by  press¬ 
ing  the  lever  under  the  triangle,  Pressing  the  correct  lever  before  the 
shock  is  considered  an  avoidance  response.  Pressing  the  correct  lever 
after  the  shock  has  started  is  considered  an  eecape  response.  Never 
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[l]  Frank  T.  Wilinski  -  Effects  of  Atropine  Sulfate  on  Trained  Monkeya 
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pressing  the  correct  lever  is  a  no  response.  The  time  interval  between  the 
trial  start  and  a  correct  response  is  considered  response  latency.  Pressing 
either  lever  when  their  is  no  figure  on  the  screen  is  called  an  intertrial 
re  sponse . 

The  electrical  equipment  associated  with  trial  presentation  and  the 
paper  tape  punch  are  rack  mounted  behind  each  booth.  Two  (2)  loops  of 
punched  mylar  tape  on  each  rack  control  the  presentation  of  the  trials. 

The  shorter  loop  initiates  trial  starts  and  is  punched  at  random  intervals 
in  order  that  no  discernible  trial  start  pattern  will  be  presented  to  the. 
monkey.  Circuity  in  the  rack  presents  the  triangle  on  the  right  or  left 
screen  in  a  random  order  with  the  restriction  that  the  long  term  expectation 
of  the  number  of  presentations  on  the  two  sides  be  equal.  The  longer  tape 
starts  and  stops  the  trial  presentation  tape.  The  monkeys  are  given  five  (5) 
sessions  per  day,  55  minutes  each,  with  a  five  (5)  minute  break  between 
sessions.  No  presentations  are  made  for  the  remaining  19  hours.  The 
monkeys  live  in  the  test  booths  for  several  days  during  testing.  Food  and 
water  are  provided  adlibitum  and  the  cage  provides  enough  room  for  the 
monkey  to  lie  down.  A  tray  of  absorbent  material  underneath  the  woven 
wire  floor  is  provided  for  excretions. 

A  record  of  the  experiment  is  made  on  punched  paper  tape  containing 
six  (6)  information  codes.  When  a  trial  is  initiated  the  punch  emits  a 
"start  of  trial  punch"  and  continues  to  run  at  10  characters  per  second, 
emitting  a  code  associated  with  latency  until  a  correct  response  is  made  or 
the  trial  is  automatically  terminated.  Separate  codes  are  made  for  latencies 
following  a  right  or  left  screen  presentation.  At  presexvt  no  distinction  is 
made  between  right  or  left  latencies  upon  computer  analysis.  A  code  is 
associated  with  the  avoidance  response,  with  the  escape  response  and  with 
an  incorrect  response.  A  separate  code  for  right  or  left  presentations  is 
provided  for  an  intertrial  response.  Since  it  is  quite  possible  that  two  or 
three  of  the  above  things  could  happen  in  a  single  l/lO  second  period,  the 
code  is  designed  for  this.  Forty-one  separate  codes  may  appear.  Since 
the  tape  has '6  information  channels  it  is  possible  to  represent  up  to  64 
codes  thus  41  presents  no  coding  problem.  At  the  end  of  a  session  an  end 
of  session  code  is  automatically  punched. 

At  the  end  of  a  day  the  punched  paper  tapes  are  removed  from  the  take- 
up  roll  on  each  equipment  rack  and  sent  to  the  computer  group  for  analysis. 
During  the  55  minute  sessions  an  average  of  104  trial  presentations  are 
made.  To  obtain  frequent  measurements  of  the  progress  of  the  monkey, 
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the  data  are  considered  as  4  subgroup*  by  the  computer,  These  subgroups 
are  termed  segments,  The  first  3  segments  contain  exactly  26  trials  while 
the  last  contains  the  number  of  trials  remaining, 

For  each  segment  and  for  the  session  the  geometric  mean  of  the  trial 
latencies  is  computed,  The  computer  determines  each  trial  latency  by 
counting  the  number  of  tape  frames  between  the  start  of  trial  punch  and  an 
avoidance  or  escape  punch,  If  neither  occur  in  100  tape  frames  this  is 
considered  a  no  response,  and  the  latency  is  taken  as  10  seconds.  The 
standard  error  is  computed  in  terms  of  log  latencies  for  each  segment  and 
each  session.  The  95%  fiducial  limits  are  computed  for  the  geometric  mean 
latency  for  each  segment  and  for  each  session,  and  the  mean  and  its  limits 
are  printed,  Analysis  in  terms  of  log  latencies  is  done  to  minimise  the 
skewness  of  the  latencies  which  results  from  the  physical  inability  of  the 
animal  to  react  in  less  them  2  or  3  tenths  of  a  second,  This  would  truncate 
the  deviations  on  the  minus  side.  Deviations  on  the  positive  side  are  only 
truncated  after  the  cut  off  time  of  10  seconds.  Since  the  mean  response 
time  is  generally  from  l/2  to  1  second  the  positive  deviations  can  be  many 
times  as  large  as  the  negative  ones.  This  causes  skewnese,  Conversion 
of  the  latencies  to  their  logarithms  minimizes  this  skewns ss. 

Session  one  of  each  day  is  considered  a  control  run  and  any  drug  la 
administered  between  session  one  and  two.  A  "t"  test  for  significance  le 
made  between  the  mean  latency  in  terms  of  logarithms  of  each  of  the  4 
subsequent  sessions  and  the  control  run.  The  statement  "not  significant, 
or  significant  at  95%,  or  significant  at  99%,  or  significant  at  99.  9%M  is 
prlntsd  after  each  of  the  sessions  2,  3,  4  and  5.  The  sum  of  all  latencies 
for  a  session  is  printed  at  the  end  of  each  session.  In  addition  to  the 
analyals  of  the  latencies  the  computer  counts  and  prints  for  sach  segment 
the  number  of  occurrences  of  each  of  the  following:  avoidance  responses, 
escape  responses,  Incorrect  responses  dons  with  the  right  hand,  Incorrect 
responses  done  with  the  left  hand,  intertrial  responses  done  with  the 
right  hand,  intertrial  responses  done  with  the  left  hand  and  the  no  response*. 
No  analysis  is  made  of  these  figures  at  the  present  time. 

A  typical  computer  printed  output  ie  presented  ae  Figure  I.  The 
numeric  portion  of  this  output  is  simultaneously  punched  into  paper  tape. 
Thie  tape  is  to  be  converted  into  Holorith  cards  for  storage  and  will  allow 
future  manipulation  of  the  test  results, 
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FATIGUE -LIMIT  ANALYSES  AND  DESIGN 
OF  FATIGUE  EXPERIMENTS 

A.  H,  Soni  and  R.  E.  Little 
Oklahoma  State  University 
Stillwater,  Oklahoma 


INTRODUCTION,  It  ie  generally  accepted  that  there  la  aa  much,  if 
not  more  acatter  aaiociated  with  fatigue  than  with  any  other  mode  of 
failure.  Conaequently,  fatigue  preaente  a  challenging  problem  to  both 
the  engineer  and  the  ctatistician. 

The  purpose  of  fatigue  analyses  is  to  adduce  information  about  the 
probability  of  relatively  rare  events,  not  to  describe  the  mean  or  modal 
event.  Accordingly,  the  statistical  problem  in  fatigue  la  to  eatablieh  the 
alternating  stress  amplitude  that  corresponds  to  the  optimum  economic 
level  of  tolerable  failures, 

A  brief  resume  of  the  nature  of  fatigue  is  presented  here  before 
discussing  existing  data  and  the  design  of  future  experiments, 

NATURE  OF  METAL  FATIGUE.  Fatigue  is  caused  by  continued  cycle 
stressing.  A  fatigue  failure  can  be  recognised  by  fitting  the  two  broken 
pieces  back  together  and  observing  the  original  geometry,  As  indicated 
in  Figure  1,  there  is  no  evidence  of  gross  plastic  deformation  prior  to 
failure  by  fatigue, 

Fatigue  cracks  are  the  cumulative  result  of  micro -inelastic  behavior 
occuring  within  the  substructure  of  the  metal.  Electron  microscopy  and 
X-ray  diffraction  studies  have  shown; 

(1)  the  physical  mechanisms  associated  with  fatigue  are  of  a 

-  3  -7 

10  to  10  cm  observation  level,  and 

(2)  these  physical  mechanisms  are  Intimately  related  to  actual 
defects  (dislocations)  in  the  theoretical  atomic  arrangement, 

The  statistical  nature  of  fatigue  is  intuitively  apparent  when  fatigue  is 
viewed  as  being  caused  by  these  minute  substructural  defects. 


ALTERNATING  STRESS  AMPLITUDE 
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Figure  1  S-N  Curve 

The  lower  the  alternating  stress  amplitude 
the  greater  the  over-all  fatigue  life,  N, 
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Thin  intuitive  view  can  be  enhanced  by  considering  an  idealised  material 
model.  First,  recall  that  metals  are  aggregate  structures  of  randomly 
oriented  crystallites  (grains),  and  that  individual  crystallites  are  anieo- 
tropic  (exhibit  differenct  properties  and  strengths  in  different  directions). 
Now  consider  the  static  yield  strength  of  the  metallic  tensile  specimen 
shown  in  Figure  2.  It  theoretically  has  a  unique  yield  strength  only  if  all 
crystallites  are  perfect  and  have  the  same  orientation.  But,  since  the 
crystallites  of  commercial  metals  have  defects  and  are  randomly  oriented, 
the  crystallites  within  this  specimen  must  exhibit  a  strength  distribution. 

Observe  in  Figure  2  that  only  a  few  crystallites  experience  yielding 
at  low  stress  levels.  But,  under  sdternating  stressing  (Figure  1),  these 
few  crystallites  yield  first  in  tension,  then  in  compression,  then  again  in 
tension,  and  so  forth.  This  localised  reversed  slip  deformation  will 
eventually  lead  to  a  fatigue  crack  in  crystallites  where  the  slip  magnitude 
(fatigue  intensity)  is  high.  Thus,  the  number  of  crystallites  that  serve 
as  potential  fatigue  crack  Initiation  sites  as  well  as  the  fatigue  intensity 
at  these  sites  are  directly  related  to  the  crystallite  strength  distribution. 
Accordingly,  fatigue  is  a  statistical  problem, 

Fatigue  failure  theories  are  in  their  infancy- ■  -theory  lags  experimental 
work,  The  present  criterion  for  the  relative  evaluation  of  various 
statistical  functions  is  simply  their  goodness  of  fit  with  regard  to  data. 
Figure  3  ehows  the  two  types  of  fatigue  data  considered,  namely: 

(1)  data  stated  in  terms  of  a  life  distribution. 

(2)  data  stated  in  terms  of  a  strength  distribution. 

In  turn,  the  over-all  objective  of  all  statistical  analyses  of  fatigue  data 
is  to  develop  the  P-S-N  surface  shown  in  Figure  4, 

Existing  data  Indicates  that  the  P-S-N  surface  is  warped  and  cannot 
be  described  in  its  entirety  by  a  simply  mathematical  function.  This 
paper  treats  a  small  but  significant  portion  of  this  surface- - -the  statisti¬ 
cal  analyses  of  fatigue -limits  in  terms  of  a  strength  distribution. 
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PART  I  -  ANALYSES  OF  EXISTING  FATIGUE -LIMIT  DATA 

COMMON  DISTRIBUTIONS.  The  three  common  statistical  functions 
applied  herein  to  fatigue -limit  data  are  listed  rows  1,  2,  and  3  of  Table  1. 

Typical  fatigue -limit  data  appears  in  Table  2.  Observe  that  the 
statistics  recorded  are  simply  the  alternating  stress  amplitudes  and  the 
corresponding  proportion  of  specimens  failed  prior  to  the  given  fatigue 
life. 

These  common  functions  are  fitted  to  the  observed  statistics  by  using 
a  minimum  residual  x^  approach.  For  example,  the  logistic  function  is 

2  .  A  2  a/s/n 

fitted  by  minimizing  the  logit  x  =  23  Npq(j£  -  £  )  ,  where  ft  =  a  +  p  s. 

2  a 

Taking  the  partial  derivative  of  the  logit  x  with  respect  to  a  and  |3  and 
then  setting  these  expressions  equal  zero;  simultaneous  solution  of  the 
two  resulting  equations  yields  the  expressions  for  the  estimates  listed 
in  rows  4  and  5  of  Table  1. 

OTHER  DISTRIBUTIONS.  The  goodness  of  fit  of  the  common  (two- 
parameter)  functions  can  be  evaluated  by  examining  the  goodness  of  fit  for 
three -parameter  functions,  i.  e.  ,  determining  whether  the  third  parameter 
is  really  required  to  describe  the  data. 

Table  3  lists  these  three -parameter  functions.  The  estimates  listed 
in  rows  4,  5,  and  6  are  established  by  taking  the  partial  derivative  of 

X23  with  respect  to  a  ,  p  and  y,  respectively;  setting  these  expressions 

equal  to  zero,  and  then  solving  these  three  equations  simultaneously. 

The  significance  of  the  third  parameter,  y,  can  be  now  determined 
from  the  magnitude  of 

2  2 
X  ~  Xn 


(x3  /  K-3) 


where  F  has  one  and  (K-3)  degrees  of  freedom.  (At  least  five  datum 
points  are  desirable  for  comparative  residual  x^  analyses.  ) 


TABLE  1.  COMMON  -DISTRIBUTIONS 
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TABLE  2,  RESULTS 

OP  R0TATIN3  BENDItC  FATIGUE 

TESTS 

ON  SAX  4340 

STEEL.  (N  . 

10’  cyclaa) 

Su  * 

190  kai, 

-  2.6. 

(Dac*  by  Cunminga,  Stulan, 

«nd  Schulta) 

SCraaa 

T«*e 

Laval 

Numbar 

Numbar 

Proportion 

a,  kai 

Taatad 

Fail. d 

Fallad 

1 

P 

32 

110 

1 

0.0091 

2 

35 

60 

3 

0.0500 

3 

38 

30 

6 

0.2000 

4 

41 

SO 

14 

0.7000 

5 

42 

so 

16 

0.8000 

tABU  3.  THREE- PARAMETER  MSTRTBtmONS 
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EXISTING  DATA  [l]  .  The  mean- square  error  associated  with  fitting 
the  two-  and  three-parameter  functions  appears  in  Table  4.  Although  the 
two-parameter  functions  are  similar,  the  logistic  and  the  extreme  value 
functions  fit  the  data  slightly  better  than  the  integrated  normal  curve. 

See  Figure  5.  In  turn,  the  three-parameter  functions  fit  the  data  some¬ 
what  better  than  the  two-parameter  functions  as  shown  in  Figure  6. 
However  the  third  parameter  is  required  for  only  about  one-half  the  data. 

Table  5  emphasizes  the  similarities  in  the  descriptive  abilities  of 
these  functions.  The  respective  (calculated)  10,  50,  and  90  per  cent 
responses  are  identical  for  practical  purposes.  These  functions  differ 
only  at  their  tails  as  indicated  by  the  extrapolated  0. 1  per  cent  response. 
(These  0.1  per  cent  responses  are  computed  only  for  illustrative  pur¬ 
poses,  and  are  not  intended  for  use  in  design.  ) 

Clearly,  these  data  are  not  adequate  to  discern  which  function,  if 
any,  precisely  describes  the  nature  of  the  fatigue -limit.  Consequently, 
further  experimental  study  is  required.  The  second  part  of  this  paper 
deals  with  the  design  of  these  tests. 


PART  II  -  DESIGN  OF  FUTURE  FATIGUE -LIMIT  EXPERIMENTS 

EXPERIMENT  DESIGN.  The  design  of  fatigue -limit  experiments  must 
overtly  reflect  efficiency  in  terms  of  over-all  cost.  Thus  it  is  imperative 
to  exploit  fatigue  testing.  In  turn,  two  considerations  are  basic  to  exploi¬ 
tation  of  fatigue  testing: 

(1)  the  minimum  number  of  specimens  required  (for  testing  at  a 
given  alternating  stress  amplitude)  to  attribute  a  prescribed 
level  of  confidence  in  the  position  of  the  datum  point,  and 

(2)  preselected  spacing  of  the  different  alternating  stress 
amplitudes  (datum  points)  to  describe  the  distribution  in 
an  efficient  manner. 

The  following  discussion  shows  how  simple  statistical  concepts  can  be  used 
to  design  more  efficient  fatigue  tests. 


P,  PERCENTAGE  FAILEO  PIRIOR  TO  107  STRESS  CYCLES 


Figure  5  Typical  Performance  of  the  Two-Parameter  Functions 


Fig  ur«  6  Typical  Performance  of  the  Three-Parameter  Function* 
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The  minimum  number  of  fatigue  specimens  required  for  testing  at  a 
given  alternating  stress  amplitude  may  be  deduced  by  considering  the 
possible  variation  in  the  observed  quantal  response.  For  simplicity, 
assume  that  the  specimen  response  is  described  by  a  binomial  distribu- 

2  PQ 

tion  that  has  parameters  P  and  cr  =  -rr  and  a  coefficient  of  variation 

p  N 

of  C.  V.  =  v  Reliable  estimates  of  P  require  a  small  C.  V,  -  -on 

NP 

the  order  of  0.  2,  Thus,  approximately  225  specimens  should  be  tested 
to  estimate  P  =  0,1.  No  such  experimental  results  are  available.  More¬ 
over  it  is  likely  that  none  will  be  forthcoming  in  the  immediate  future 
because  this  test  alone  could  cost  up  to  $10,000.  (A  single  fatigue 
machine  running  at  10, 000  RPM  night  and  day  would  take  eight  years  to 
complete  such  a  test  if  the  desired  fatigue  life  is  5  X  10®  cycles). 

Clearly,  statistical  efficiency  must  be  sacrificed  in  fatigue-limit 
tests.  A  coefficient  of  variation  on  the  order  of  0.  5  la  probably  the  best 
that  csui  be  expected.  Even  then,  approximately  400  specimens  are 
required  to  estimate  P  =  0.01.  Thus,  it  appears  that  ths  coefficient  of 
variation  approach  to  deducing  the  number  of  fatigue  specimens  required 
in  testing  will  satisfy  neither  the  statistician  nor  the  materials  analyst. 

It  is  possible  to  mitigate  this  problem  somewhat  by  estimating  the 
number  of  specimens  required  by  a  different  approach,  vie,  ,  selecting 
N  ouch  that  the  parameters  have  a  negligible  bias,  The  logistic  function 
is  selected  here  to  illustrate  this  approach,  (This  selection  is  made  on 
the  basil  of  ease  of  computation.  .  .  .there  is  relatively  little  difference 
in  the  descriptive  abilities  of  any  of  the  functions  considered  here  within 
the  probability  ranges  of  existing  fatigue -limit  data,) 

The  linear  transform  of  the  logistic  function  is  given  by 
(1)  l  =  a  +  (is  +  < 

where  <  is  the  (random)  error  associated  with  £  .  This  transform  is 
used  to  fit  the  logistic  function  to  the  data,  However,  to  accomodate  sub¬ 
sequent  graphical  solution  of  (3  ,  this  transform  is  temporarily  redefined 
as  [2, 3] 
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£'  =  In 


P  + 


1 

2N 


Q  + 


1 

2N 


a  +  0s  +  «, 


The  error  and  variance  of  are  given  by 


(3) 


+  E  (£'  -  a  -  (3s)  =  e"Npq  [Npqln3  +  -yy  (Npq)2ln5 

+  37  (Npq)3  ln7  +  .  ,  -  In  (2Npq) 


V(jn=e'Npq  (Npq(ln3)2  +  ^7  (Npq)2(ln5)2  +  .  . } 


(4) 


••  '2Npq  (Npqln3  +  (Npq)2ln5  + 
and  the  asymtotic  mean  and  variance  ere: 


(5) 

(6) 


E(f ')  *  a  +  (3s 

1 


v(r)  = 


Npq 


Thus,  it  is  clear  that  bias  is  a  function  of  Npq,  This  relationship  is 
shown  in  Figure  7,  where  it  can  be  seen  that  a  value  of  Npq  of  two  or 
larger  affords  unbiased  estimates  of  the  populations  parameters, 
Accordingly,  the  minimum  number  of  specimens  required  at  a  given 
alternating  stress  amplitude  can  be  read  from  Figure  8. 

The  spacing  of  the  different  alternating  stress  amplitudes  should 
be  sufficiently  wide  to  attain  an  efficient  estimate  oi  (3  ,  Considering 
the  logistic  function: 


(7) 


£ 1  -  i ' 
X1  *2 


fi* 


I 
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where  i}'  >  l'?\  sL  >  and(«1  -  s  )  =  d,  the  spacing, 
Equation  7,  restated  in  terms  of  d,  becomes 


Now,  selecting  p^  such  that 


(9)  ?!  -p2>  1  VHHi  +  (-^-)2 

this  inequality  can  be  rewritten  as 


(10) 


P2<P1 


Finally,  substitution  of  Equation  (10)  into  Equation  (8)  gives  the  desired 
spacing 


when  Npq  =  2.  This  spacing  is  shown  in  Figure  9.  Note  that  the  spacing 
can  be  qualitatively  deduced  from  Equation  9  which  indicates  that  p^  - 
can  approach  zero  as  N  becomes  very  large. 


HYPOTHETICAL  FATIGUE  TEST,  Suppose  that  the  materials  analyst 
has  only  100  AISI-1020  annealed  steel  specimens  (Ultimate  Strength  =  70  ksi), 
but  wishes  to  obtain  the  most  information  concerning  the  strength  distribu- 
tion.  Figure  10  suggests  a  trial  value  of  the  alternating  stress  amplitude 


(j  ,  OBSERVED  RESPONSE 


Figure  9a  Relationihip  Between  p^  and  pB  for  Efficient  Eatimation 


{}  .OBSERVED  RESPONSE 


Figure  9b  Ralationehip  Between  and  l't  for  Efficient  Eitltnatlon  of 
The  valuaa  of  L{  and  corraapond  to  p;and  pa,  teapactlvaly. 
Obaarva  that  d  ,  >  1,75/0. 


40  80  120  160  200  240 

Su (ULTIMATE  STRENGTH,  K8I 


Figure  10  Trial  Value  for  Firat  Preliminary  Teat 
Diagram  taken  from  S^-S^  relationehlp  developed 

for  steel  by  Bullene  [5]  . 
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that  corresponds  to  a  P  of  roughly  0.  50  to  0.  75.  Using  this  trial  value, 

10  specimens  are  tested  at  S  =  42  ksi  and  it  is  observed  that  7  specimens 

7  a 

fail  prior  to  10  cycles.  Setting  d  =  5  ksi  (based  on  Figure  10),  the  second 
test  is  conducted  at  S  =  37  ksi.  In  this  second  test,  only  4  of  the  10 

specimens  tested  fail.  The  required  spacing  in  subsequent  tests  can  now 
be  determined  by  estimating  |3  (using  Equation  7).  In  this  hypothetical 
test 


Thus,  d  is  taken  as  7  or  8  and  the  number  of  specimens  is  established  as 
indicated  in  the  following  table: 


Test: 

Alternating 

p  estimated  by 

Corresponding  Adjusted 

Stress 

graphical  solution 

N 

N 

Amplitude 

(d-7) 

(P=  .225) 

for 

Npq=2 

(Npq-;1.  75) 

Fourth 

23 

0.  04 

52 

45 

Third 

30 

0.15 

16 

15 

(Second)a 

(37) 

(0.40) 

(10) 

(10) 

(First)a 

(42) 

(0.  70) 

(10) 

(10) 

Fifth 

49 

0.  90 

22 

20 

(a)  preliminary  tests 

trial _ 
total 

110 

adjusted 

* \  ,  =  100 
total 

Note  that  Npq  is  greater  than  two  for  the  preliminary  tests.  Actually 
10  specimens  are  not  required  in  either  case.  The  weights  can  be 
calculated  as  these  preliminary  tests  progress  and  the  next  test  can  be 
started  when  the  respective  Npq  approaches  two.  Then,  the  "saved" 
specimens  can  be  tested  at  the  most  appropriate  stress  amplitude  at  the 
conclusion  of  the  over-all  test. 

The  over -all  test  data  are  then  listed  in  tabular  form  (Table  2)  and 
fitted  as  outlined  in  Part  I  (Tables  1  and  3.  ) 
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SUMMARY.  Fatigue  data  will  always  be  somewhat  limited  because 
fatigue  tests  are  expensive.  Thus,  it  is  necessary  to  design  fatigue  tests 
to  be  statistically  more  efficient.  This  means  that  care  must  be  given  to 
the  preselection  of  the  number  of  specimens  tested  and  to  the  spacing 
of  the  respective  alternating  stress  amplitudes  considered. 

Present  analyses  can  only  compare  the  relative  performance  of 
different  functions  with  regard  to  goodness  of  fit  of  limited  ranges  of 
data. 
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U.  S,  Army  Aviation  Materiel  Command 
St.  Louie,  Missouri 


INTRODUCTION.  The  idea  for  this  presentation  came  as  a  result  of 
unsuccessful  attempts  to  solve  an  analytical  problem  which  was  compli¬ 
cated  by  restrairtsplaced  on  the  collection  of  data  for  analysis.  Figure  1. 
This  situation  is  not  an  isolated  one  but  generally  occurs  when  much  data 
are  already  being  gathered  and  they  are  not  sufficient  for  the  analysis 
desired.  Alteration  of  the  existing  data  collection  system  juBt  to  satisfy 
the  needs  of  a  supposedly  isolated  and  parochial  study  effort  is  generally 
not  feaeible,  So,  it  is  necessary  to  consider  the  existing  data  limitations 
as  part  of  the  problem  to  be  solved. 

In  this  case,  the  success  of  the  analytical  effort  depends  on  the 
relationship  which  is  established  between  the  kind  and  amount  of  informa¬ 
tion  which  is  needed  to  define  the  problem  and  the  kind  and  amount  of 
information  available  for  solving  the  problem  as  defined.  When  this 
prohlem-defining  and  solving  effort  does  not  provide  meaningful  results 
(Figure  2),  three  questions  are  appropriate:  has  the  problem  been 
inadequately  defined  because  of  ignorance  about  the  nature  of  the  opera¬ 
tion  being  considered?;  are  the  data  collected  not  sufficient  in  kind  and/or 
quantity  to  establish  the  desired  relationships  ?;  and  are  the  data  being 
inadequately  analyzed  because  of  the  ignorance  of  the  analysts  ?  It  is 
generally  necessary  to  assume  that  data  collected  for  analysis  are  not 
erroneous  to  the  extent  that  they  would  be  the  principal  cause  for  the  lack 
of  meaningful  analytic  results  because  it  is  seldom  feasible  to  double 
check  the  correctness  of  the  data. 

ONE  EXAMPLE.  To  illustrate  the  foregoing  remarks,  a  recent  study 
will  nowbe  described.  To  appreciate  the  need  for  this  study,,  it  is  neces¬ 
sary  to  point  out  that  AVCOM's  supply  effort  relate*  to  keeping  Army 
aircratt  from  being  too  often  deadlined  due  to  a  lack  of  parts  (commonly 
referred  to  as  an  Equipment  Deadlined  for  Parts  or  briefly  an  EDP 
situation)  while  incurring  no  more  than  the  least  costs  necessary  to  obtain 
such  results.  It  was  recognized  that  this  effort  might  be  made  more 


*The  views  contained  herein  have  not  been  approved  by  the  Department 
of  the  Army,  and  represent  only  the  views  of  the  author, 
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effective  and/or  efficient  if  it  could  be  analytically  demonstrated  how  the 
rate,  at  which  aircraft  are  EDP,  varies  with  various  supply  actions  and 
ultimately  with  the  costs  associated  with  each  action. 

A  study  to  obtain  the  desired  analytical  results  was  developed. 

a,  Concept:  It  was  recognized  that  the  total  time  that  aircraft 
are  EDP  is  affected  by  how  often  an  EDP  situation  occurs  and  how  long 
it  takes  to  satisfy  each  EDP  situation.  Therefore,  the  study  was  natu¬ 
rally  subdivided  into  a  study  of  the  frequency  of  occurrence  of  EDP 
situations  and  a  study  of  the  time  required  to  satisfy  EDP  situations. 
Immediately,  obstacles  were  encountered, 

(1)  Frequency  of  Occurrence:  How  often  each  aircraft  is  EDP 
during  the  month  Is  not  reported,  However,  when  an  aircraft  is  EDP  for 
a  part  which  is  supposed  to  be  furnished  by  AVCOM  action  and  that  part 
cannot  be  obtained  below  depot  level,  an  EDP  requisition  is  sent  to  AVCOM. 
Therefore,  there  are  at  least  aB  many  instances  of  aircraft  EDP  as  there 
are  valid  EDP  requisitions  received  at  AVOCM  and  it  is  operationally 
known  that  there  are  more  euch  occurrences  since  aircraft  are  EDP  for 
parts  which  are  supplied  without  AVCOM  action.  The  term  valid  EDP 
requisitions  is  needed  because  those  for  common  hardware  parts  which 
could  not  render  an  aircraft  operationally  deadlined  were  excluded  as 
being  invalid. 

Fortunately,  the  total  time  that  each  aircraft  is  EDP  is 
reported  to  AVCOM.  So,  it  was  hoped  that  an  estimate  of  the  amount  of 
change  which  might  be  achieved  in  the  days  aircraft  are  EDP  by  a  supply 
action  which  might  reduce  the  rate  at  which  EDP  requisitions  occur  at 
AVCOM  by  a  particular  amount,  might  be  obtained  by  regression  analyses 
by  aircraft  type.  The  results  of  these  analyses  will  be  indicated  later, 

(2)  Time  to  Satisfy:  Meanwhile,  attempts  to  relate  the  time 

to  satisfy  an  aircraft  EDP  situation  encountered  similar  data  constraints, 
When  an  aircraft  EDP  situation  is  satisfied  without  an  AVCOM  action  in 
response  to  an  EDP  requisition,  the  time  required  to  obtain  such  satis¬ 
faction  is  unknown  at  AVCOM.  So,  it  was  necessary  to  assume  that  such 
instances  have  a  random  effect  on  the  total  time  aircraft  are  EDP  in  a 
month.  Then,  a  meaningful  correlation  might  be  discovered  between  the 
time  aircraft  are  EDP  and  the  time  required  to  satisfy  an  EDP  requisi¬ 
tion  at  AVCOM. 
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Also,  the  complete  time  required  to  satisfy  an  EDP  requisi¬ 
tion  at  AVC.OM  could  not  be  easily  obtained.  The  time  that  wu  obtained 
is  the  time  between  the  date  an  EDP  requisition  is  initiated  and  the  date 
on  which  materiel  release  at  the  appropriate  depot  is  confirmed. 

In  other  words,  the  time  consumed  after  a  material  release 
confirmation  is  sent  to  AVCOM  and  until  the  part  arrives  at  the  site  of 
the  particular  EDP  aircraft  was  not  readily  measurable  and  had  to  be  left 
out  of  the  study.  Again,  it  was  necessary  to  assume  that  the  effect  of 
this  time  or.  the  total  time  aircraft  are  EDP  is  random  and  that  a  meaning¬ 
ful  correlation  might  exist  between  the  time  aircraft  are  EDP  and  the 
principal  portion  of  the  time  to  satisfy  an  EDP  requisition  measured  in 
this  study. 

On  the  other  hand,  since  an  EDP  requisition  does  not  identify 
the  specific  aircraft  which  is  awaiting  the  part,  it  is  possible  that  an  EDP 
aircraft  has  been  made  serviceable  by  using  a  part  obtained  from  some 
other  source  such  as  off  of  a  crash  damaged  aircraft  and  yet  the  pertinent 
EDP  requisition  is  not  satisfied.  It  was  hopefully  assumed  that  such 
instances  might  compensate  for  some  of  the  excluded  shipping  time. 

b.  Sample  Selection:  By  now,  hopes  to  obtain  fruitful  analytical 
results  were  waning  and  yet  the  worse  was  yet  to  come.  Since  it  was 
desired  to  obtain  some  useful  results  as  soon  as  possible  and  the  informa¬ 
tion  about  aircraft  days  EDP  is  available  only  on  a  monthly  basis,  six 
months  data  or  six  data  points  were  chosen  for  analysis,  After  the  data 
were  gathered,  there  was  reason  to  believe  that  data  concerning  all  EDP 
requisitions  received  by  AVCOM  during  the  first  three  months  observed 
had  not  been  obtained.  Further,  it  could  not  be  determined  whether  the 
sample  EDP  requisitions  could  be  validly  claimed  to  be  a  representative 
sample.  Therefore,  only  the  latter  three  month's  data  were  used  for 
regression  analysis.  At  this  point,  the  problem  being  described  can  be 
summarized  as  shown.  Figure  3. 

c.  Results  Obtained:  Approximately  nine  months  elapsed  before 
efforts  to  obtain  the  preferred  analytical  results  were  exhausted,  A 
total  of  14  aircraft  types  were  considered,  Needless  to  say,  the  results 
obtained  were  disheartening  even  though  not  unexpected, 


368 


Design  of  Experiments 


(1)  Frequency  of  Occurrence:  Table  1  contains  estimates  of  the 
relationship  between  the  days  aircraft  are  EDF  *mi  Lhe  quantity  of  EDP 
requisitions  received  at  AVCOM, 

(2)  Time  to  Satisfy;  Table  2  contains  estimates  of  the  relation¬ 
ship  between  the  days  aircraft  are  EDP  and  the  major  portion  of  the  time 
to  satisfy  EDP  requisitions  at  AVCOM. 

(3)  It  is  recognized  that  three  data  points  are  not  enough  to 
preclude  apparently  conflicting  results  from  occurring  because  of  sampl¬ 
ing  variations  but  there  were  no  more  reliable  data  points  which  could 

be  used  to  reduce  this  likelihood,  However,  the  occurrence  of  both 
positive  and  negative  correlation  coefficients  is  disconcerting,  In  the 
case  of  negative  ones,  it  is  implied  that  a  reduction  in  aircraft  days  EDP 
can  be  obtained  by  increasing  the  frequency  of  occurrence  of  EDP  instances 
or  by  taking  more  time  to  satisfy  EDP  requisitions.  Both  of  these 
implications  are  unreasonable.  With  the  hope  that  the  three  questionable 
data  points  might  be  good  ones,  regression  analyses  using  six  points  were 
made  but  no  more  reasonable  results  were  obtained, 

(4)  To  preclude  some  wrong  implications,  it  must  be  pointed 
out  that  this  nine  month  study  effort  did  not  consume  much  more  than  one 
analyst's  time.  The  study  time  had  to  take  six  months  to  obtain  six 
months  of  data.  Additional  time  was  required  to  allow  EDP  requisitions 
received  near  the  end  of  the  sixth  month  to  be  satisfied.  In  addition, 
several  by-product  analyses  were  made  with  the  data  collected.  In  other 
words,  it  would  be  unfair  to  conclude  that  this  analytical  effort  was  not 
worthwhile.  Also,  it  seems  that  it  could  be  concluded  that  the  desired 
results  were  not  obtained  for  at  least  the  first  two  of  the  reasons  listed 
on  Figure  2;  namely,  inadequate  representation  of  the  problem  and 
insufficient  data  collected  both  in  type  and  quantity, 

d.  Question:  However,  the  question  still  remains;  What  can  ^>e 
done  to  increase  the  effectiveness  of  the  analytical  effort  being  expended 
in  the  manner  just  described? 

ANOTHER  EXAMPLE,  Before  attempting  to  present  any  subjective 
answers  to  the  question  just  stated,  another  analytical  problem  area  can 
be  used  to  suggest  that  there  is  a  related  question  that  also  needs  answering, 
This  analytical  problem  is  suggested  by  a  review  of  budgeting  and  funding 
practices, 
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It  is  not  necessary  to  know  the  exact  budgeting  and  funding  procedures 
to  appreciate  the  features  which  are  useful  here.  Figure  4. 

a.  The  preparation  of  a  budget  must  be  in  accordance  with  guidance 
furnished  by  higher  headquarters.  This  guidance  has  usually  been  differ¬ 
ent  from  year  to  year.  This  situation  implies  that  a  generally  sound 
budgeting  procedure  has  not  yet  been  determined. 

b.  In  addition,  forecasted  budget  requirements  are  never  completely 
honored.  Somewhere  up  the  line,  limitations  are  set  below  the  accumu¬ 
lated  forecasted  requirements  and  these  limitations  are  somehow  partitioned 
and  passed  down  to  each  organization  involved. 

c.  Further,  each  organization's  general  objective  is  to  make  commit¬ 
ments  nearly  equal  the  limitations  appropriate  at  the  time  of  each  within 
year  review.  In  other  words,  if  there  is  only  a  mid-year  review,  commit¬ 
ments  should  be  nearly  equal  to  one  half  of  the  annual  limitation  otherwise 
it  might  be  concluded  that  even  less  funds  will  suffice  and  limitations  will 
be  decreased  accordingly.  As  a  result  of  these  within  year  reviews, 
particular  fund  limitations  for  the  remainder  of  the  year  are  revised; 
sometimes  upward  and  sometimes  downward, 

d.  At  this  point,  it  is  well  to  hypothesize  the  logic  which  supports 
this  budgeting  and  funding  practice.  It  is  initially  assumed  that  no  one  can 
forecast  an  organization's  budgetary  requirements  more  accurately  than 
the  organization  itself.  Therefore,  forecasted  requirements  are  made 

by  each  organization  and  these  are  the  starting  point  for  the  budgeting  cycle. 
Since  fund  limitations  have  always  been  set  less  than  forecasted  budget 
requirements,  organizations  find  it  expedient  to  compensate  for  such 
reduction  by  somehow  inflating  estimates  of  requirements.  It  seems  rea¬ 
sonable  to  assume  that  the  extent  of  this  inflation  cannot  be  accurately 
determined  by  the  people  who  set  limitations  otherwise  budgetary  guidance 
could  preclude  such  inflation  and  forecasted  requirements  could  be  honored. 
Also,  since  the  practice  of  setting  limitations  less  than  forecasted  require¬ 
ments  has  never  been  considered  responsible  for  serious  operational  short¬ 
ages,  the  practice  has  been  continued  without  fear. 

It  seems  that  this  strategic  exercise  must  persist  until  it  has  been 
definitely  learned  that  the  allotment  of  different  quantities  of  funds  leads 
to  the  achievement  of  recognizably  different  accomplishments.  Only  then 
can  superiors  choose  the  desired  amount  of  accomplishments  and  fund 
accordingly.  Thus,  the  question  arises; 
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How  can  the  regression  analysis  effort,  necessary  to 
establish  a  sound  relationship  between  money  allotted  and 
results  achieved  thereby,  be  obtained? 

i  . CONCLUDING  REMARKS,  In  review,  it  seems  that  the  two  aituations 
just  described  indicate  a  need  for  a  way;  of  improving  the  effectiveness  and 
efficiency  of  the  analytical  effort  trying  to  do  regression  analysis  in  a 
subordinate  command  such  as  AVCOM;  and  of  getting  regression  analysis 
implemented  in  a  higher  headquarters  in  the  budgeting-funding  subject  area. 

a.  In  the  first  case,  it  is  possible  to  take  the  viewpoint  that  certain 
analytical  efforts  should  be  dropped  when  data  collection  restraints  are 
too  restrictive  or  that  it  is  worthwhile  to  expend  some  effort  to  remove 
as  many  of  those  restraints  as  necessary,  However,  the  potential  value 
of  certain  analytical  results  is  sufficient  to  preclude  their  being  dropped 
until  it  has  been  indisputably  demonstrated  that  they  cannot  be  obtained 
in  spite  of  existing  constraints,  Also,  the  removal  of  data  collection 
restraints  to  satisfy  local  analytical  needs  is  practically  impossible  since 
existing  data  collection  and  reporting  requirements  have  been  entrenced 
by  tradition  and  austerity  measures  in  the  manpower  area  preclude  the 
collection  and  reporting  of  additional  data  for  local  analyses  that  have  not 
been  specifically  required  by  higher  headquarters.  Therefore,  it  seems 
that  some  outside,  authoritative  intervention  is  needed  if  the  situation 
confronting  local,  investigative  analyses  is  to  be  improved, 

b.  In  the  second  case,  since  budgeting  guidance  is  furnished  by 
higher  headquarters  and  munt  be  adhered  to  by  subordinate  commands, 

it  seems  that  regression  analysis  must  be  attempted  and  found  successful 
at  the  top  before  the  official  authorisation  to  do  such  analysis  at  the  bottom 
esm  be  expected  and  before  the  cooperation  necessary  to  have  a  reasonable 
chance  at  being  successful  with  this  effort  will  be  forthcoming, 

In  other  words,  it  seems  that  it  is  not  enough  to  hire  analysts  at  all 
levels  in  *he  Department  of  the  Army  and  then  allow  organisational  tradi¬ 
tion  to  render  such  analysts  ineffective  and  inefficient.  The  situation  could 
be  significantly  improved  if  (Figure  5)  the  Office  of  the  Chief  of  Research 
and  Development  (OCRD)  would  form  a  Survey  Team  of  renowned  analysts 
who  would  visit  selected  Army  headquarters  to  determine  the  extent  and 
kind  of  analytical  program  that  seeme  appropriate  for  each  organisational 
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level  and  would  then  prepare  a  recommended  Department  of  the  Army 

program,  Then.OCRD  could  coordinate  this  nm ».  - - - 

«na  airect  that  the  coordinated  program  be  done.”  This  typTof  positive 
a  biJ  extreme  and  probably  impo.aible  to  obtain  and  so  a 

the  .  ft  l0r/°r/  698  extreme  improvement  action  and  one  more  within 
the  authority  of  a  subordinate  organization  is  hereby  extended 


A/C 

Table  1 

A/C  Days  EDP  vs 

Correlatioi 

las. 

Qty  of  EDP  Rqni 

Coefficient 

OH-13 

y  =  1921  +  0,  ?7?x 

0,  927 

UH-19 

y  =  397  +  2.  61  Ox 

0.  903 

CH-21 

y  =  -419  +21,  17?x 

0,  933 

OH-23 

y  =  2701  -  0,  890k 

-0,  120 

CH-34 

y  =  399  +  5, 472x 

0,  621 

CH-37 

y  =  588  -  5, 44?x 

-0.  683 

UH-1 

y  »  1572  +  1,  67 9x 

0,  415 

0-1 

y»  519  +20, I07x 

0,  783 

U-6 

y  ■  908  -  1,  54 3x 

-0,  347 

U-8 

y  ■  533  -  2,  40 3x 

-0.  713 

U-l 

y  •  391  ■  1.  582x 

-0.  281 

OV-1 

y  ■  468  +  2,  61  Ox 

0,  976 

CV-2 

y  ■  362  -  0,  522x 

-0,  649 

CH-47 

y  ■  883  -  5,  56?x 

-0,  997 

y  “  is  in  terms 

of  aircraft  days  EDP 

x  ■  is  in  terms  of  quantity  of  EDP  requisitions  received  at  AVCOM 
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A/C 

Type 

Table  2 

A/C  Day*  EDP  v* 

Qty  of  EDP  Rc^n* 

Correlation 

Coefficient* 

OH-13 

y  =  1923  +  0.  12  6x 

0,  998 

UH-19 

y  =  567  +  0.103x 

0,  394 

CH-21 

y  «  509  -  0,  04 6x 

-0, 088 

OH-23 

y  =  3194  -  0,  457x 

-0, 662 

CH-34 

y  »  482  +  0, 382x 

0,  777 

CH-37 

y  »  577  -  0.  220x 

-0,  883 

UH-1 

y  ■  6913  -  1,  551x 

•0. 388 

0-1 

y  a  3151  -  6.  U9x 

-0.498 

U-6 

y  ■  950  -  ,  184x 

-0,465 

U-8 

y  •  56  +  0,  5 3 Ox 

0,999 

U-l 

y  ■  302  +  ,  116x 

0,  681 

OV-1 

y  •  431  +  0, 132x 

0,  619 

CV-2 

y  ■  359  -  0,  023x 

-0, 728 

CH-47 

y  ■  690  -  0, 092x 

-0. 566 

y  B  i*  in  term*  of  aircraft  day*  EDP 

x  ■  i*  in  term*  of  the  principal  quantity  of  day* 

required  to  *ati*fy  LDP  requisition*  received  at 
AVCOM 
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Problem  To  Be  Solved 

Analytical 

Problem 

Plus 

Data  Collection 
Restraints 

FIGURE  1 


No  Useful  Solution  Obtained 
Inadequate  Representation  of  Problem? 
Inefficient  Data  Collected? 

Inadequate  Analysis? 

FIGURE  2 


Sample  Problem 

Effect  of  Supply  Actions  on 
Acft  EDP  Rate 

Plus 

Inexact  EDP  Frequency 

Unknown  Extent  of  EDP  Change 
Due  to  Non-AVCOM  Action 

Unknown  Shipping  Times 

Only  Three  Reliable  Data  Points 


FIGURE  3 
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■lj  oa  a. 

Guidance  Changes  Annually 

Limitations  Less  Than 
Forecasted  Requirements 

Commit  Full  Limitations 

Within  Year  Reviews 

Revised  Limitations 

FIGURE  4 


For  Consideration 
PROFESSIONAL  TEAM: 

Conduct  Survey 

& 

Describe  Analytical  Program 
CHIEF,  RESEARCH  fc  DEVELOPMENT: 
Require  Program  Be  Done 


FIGURE  5 


ASSESSMENT  AND  CORRECTION  OF  DEFICIENCIES  IN  PERT 


II.  O.  Hartley  sad  A  w  Wortham 
Institute  of  Statistics 
Texas  A&cM  University 
College  Station,  Texas 


1,  INTRODUCTION.  As  is  well  known,  the  technique  known  under  the 
name  of  PERT  (Program  Evaluation  Review  Technique)  is  concerned  with 
a  'project'  comprising  a  large  number  of  successive  'activities'  which  are 
arranged  in  a  complex  'network'  (see  e,  g.  Figure  2).  Each  activity 
'commences'  at  a  particular  'point'  of  the  network  but  not  until  all  activities 
'terminating'  at  that  point  are  completed.  Specifically,  PERT  is  concerned 
with  computing  the  expected  time  required  to  complete  all  activities  of  the 
project:  -Assuming  that  the  time  taken  to  complete  a  particular  activity 
follows  a  specified  diatribution  of  completion  times,  the  total  time  needed 
to  complete  the  project  the  so  called  'critical  time'  ia  a  statistical  vari¬ 
able  and  is  given  by  the  total  of  completion  times  along  the  'critical  path', 
i,  e.  along  that  sequence  of  aetivitiea  in  the  network  which  for  a  given 
sample  of  completion  times  takes  longest  to  reach  every  point  along  its 
path.  The  expected  value  of  this  critical  timo  is  the  expected  time  to  com¬ 
plete  the  project, 

Now  it  ie  well  known  that  PERT  does  not  compute  the  correct  critical 
time  as  defined  above  but  instead  uees  for  each  activity  the  average  com¬ 
pletion  time  and  then  determines  a  unique  and  fixed  critical  path  as  the 
sequence  of  aetivitiea  for  which  the  sum  of  the  expected  completion  times 
is  at  a  maximum,  Critical  path  determination  by  this  method  may  be 
badly  misleading  and  may  result  in  a  serious  underestimate  of  the  expected 
time  to  complete  the  project.  Moreover,  It  may  also  lead  to  erroneous 
information  on  the  Identification  of  'critical  activities',  i.  e.  ,  activities 
which  are  crucially  responsible  for  the  delay  in  completion  of  the  project. 

Whilst  this  shortcoming  of  PERT  has  been  known  from  its  initiation 
and  the  above  method  is  deliberately  used  as  an  approximate  short-cut, 
we  do  not  think  that  the  magnitude  of  the  bias  in  this  short-cut  method  is 
fully  appreciated,  Indeed  it  can  be  shown  (see  e,  g.  section  8)  that  under 
certain  circumstances  PERT  may  underestimate  the  correct  expected 
completion  time  by  50%  or  more.  Moreover,  for  a  general  network,  PERT 
provides  the  correct  answer  only  under  the  (completely  unrealistic) 
assumption  that  there  ie  essentially  no  variability  in  the  completion  times 
for  each  activity. 
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One  of  the  objectives  of  this  paper  is  therefore  to  eliminate  this  bias 
from  PERT;  in  fact,  we  shall  provide  a  method  of  computing  the  probability 
distribution  of  critical  times  and  thereby  supply  not  only  the  correct  value 
of  its  expectation  but  likewise  of  its  variance  and  percentage  points. 

It  may  rightly  be  argued  that  our  exact  method  of  critical  path  analysis 
is  based  on  the  assumed  distribution  of  completion  times  for  each  activity, 
and  that  there  is  usually  a  notorious  lack  of  information  on  such  timings. 
This  point  is  well  taken.  However,  we  feel  that  the  deplorable  lack  of 
input  data  should  not  excuse  us  from  using  a  method  accurately  utilizing 
at  least  all  the  available  information.  Moreover  and:  more  positively  our 
method  enables  the  analyst  who  is  uncertain  about  the  completion  times  of 
(say)  a  particular  activity  in  the  network  to  evaluate  the  effect  of  altering 
his  assumptions  about  that  activity  on  the  critical  time  and  path.  We 
consider  the  provision  of  such  a  'sensitivity  analysis  of  PERT'  as  an 
important  contribution  to  planning  a  project  'under  uncertainty'. 

Mathematically  our  method  utilizes  the  following  devices;  - 

(a)  A  classification  of  networks  into  different  types  depending  on  their 
degree  of  involvement  and  complexity. 

(b)  An  operational  calculus  by  which  the  distribution  of  critical  times 
will  be  derived  by  numerical  analysis,  notably  numerical 
integration.  This  method  will  provide  the  solution  to  our  prob.^- 
lem  for  the  basic  types  of  networks. 

(c)  A  Moi  Carlo  procedure  providing  an  approximate  solution  for 
the  more  involved  networks. 

(d)  Analytic  solutions  for  particularly  simple  networks  and  partic¬ 
ularly  simple  distributions  of  completion  times.  These  are 
mainly  used  for  illustration  purposes. 

2.  GENERAL  DEFINITIONS  AND  'UNCROSSED  NETWORKS'.  In  order 
to  provide  a  mathematically  rigorous  theory  of  PERT  analysis  for  networks, 
it  is  necessary  to  introduce  certain  definitions  and  concepts.  We  therefore 
give  the  following  definitions  and  explanations;  - 
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2.1.  An  activity  is  represented  by  one  or  two  line  segments  in  the 

network  (see  Figure  1).  It  'commences1  at  one  of  its  ringed  end 
points  and  'terminates'  at  the  other  ringed  end  point,  the 
'direction  of  the  time  flow  being  indicated  by  the  arrow.  The 
numbering  of  the  activities  is  explained  in  2.  3. 

2.  2.  A  Network  Point:  -  These  are  represented  by  ringed  points  in 
Figure  1.  A  network  point  represents  any  stage  in  the  network 
occurring  at  the  beginning  and/or  end  of  an  (or  several) 
activity(ies)  (e.  g.  ,  event  5  in  Figure  1  is  a  network  point 
since  activity  (7;  2,  5)  terminates  and  activities  (10;  5,8)  and 
(11;  5,8)  commence  at  that  stage  of  the  network. 

2.  3.  Codes:  -  'Network  points'  carry  a  serial  number  (ringed  in 
Figure  1)  identifying  them.  The  order  of  the  numbering  is 
immaterial  at  this  stage.  An  activity  also  carries  a  'serial 
number'  (preceding  the  ;  )  but  also  the  number  of  the  network 
point  at  which  it  commences  followed  by  the  network  point 
number  at  which  it  terminates.  Thus  (7;  2,  5)  denotes  activity 
No.  7  commencing  at  point  No.  2  and  terminating  at  point 
No.  5. 

2.4.  Two  consecutive  activities  are  defined  as  activities  numbered 
(t;  i,  j)  and  (s;  j,k)  i.  e.  ,  the  first  terminates  at  point  j  whilst 
the  second  commences  at  point  j. 

2.  5.  A  Path  from  i  to  j  is  a  'sequence  of  consecutive  activities' 
starting  at  point  i  and  finishing  at  point  j  (e.  g.  ,  activities 
(2;  0,2),  (7;  2, 5)  and  (10;  5,  8)  starting  at  point  (0)  and  terminat¬ 
ing  at  point  (8)). 

2.  6.  A  complete  path  -  A  path  starting  at  the  beginning  and  finish¬ 
ing  at  the  end  of  the  project  (e.  g,  ,  the  path  formed  by  (l;  0,1), 

(5;  1,4),  (9;  4,7)  and  (15;  7,10))  . 

2.7.  A  Universal  Point  -  A  network  point  through  which  all  complete 
paths  pass  (the  only  universal  points  in  Figure  1  are  at  0  and  10). 

2.8.  Consecutive  Points  -  Point  j  is  consecutive  to  Point  i  if  both  j 
and  i  are  universal  and  if  all  paths  starting  from  i  pass  through 
j  before  passing  through  any  other  universal  point  (if  any). 


(17|9,10) 


FIGURE  1 

NRTVORX  NOTATION 
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2.9.  Sets  of  first  order  branches  -  Consider  the  set  of  all  paths  com¬ 
mencing  at  a  universal  point  i  and  terminating  at  a  universal 
point  j  consecutive  to  i.  Subdivide  the  set  of  these  paths  into 
exhaustive  subsets  such  that  any  two  paths  in  different  subsets 
have  only  points  i  and  j  in  common  but  any  two  paths  in  the  same 
subset  have  at  least  one  more  point  in  common.  (This  is  always 
possible  since  we  may  place,  if  necessary,  all  paths  in  the  same 
subset.)  These  mutually  exclusive  subsets  are  called  '1st  order 
branches.  '  (e,  g.  ,  in  Figure  1  the  paths  formed  by  connecting 
points  0,1,4,7,10  form  the  first  1st  order  branch,  the  paths 
formed  by  connecting  points  0,2,  5,8,10  the  second  1st  order 
branch  and  the  paths  formed  by  0, 3,  6,9.10  the  third  1st  order 
branch.  )  If  there  are  only  two  consecutive  points  in  the  network 
(i.  e.  ,  the  start  and  the  end)  s.nd  there  is  only  one  set  of  paths 
as  described  above,  we  shall  term  it  a  zero  order  branch.  For 
example,  Figure  3  would  constitute  a  single  zero  order  branch, 
so  would  a  single  activity  network. 

2.10.  Sets  of  2nd  order  branches  -  Consider  a  particular  1st  order 
branch  starting  at  a  universal  point  i  and  ending  at  a  universal 
point  j  consecutive  to  i  and  regard  it  as  a  separate  network. 
Apply  definitions  2. 1  to  2.  9  to  this  network,  then  any  1st  order 
branches  of  this  first  order  branch  are  called  second  order 
branches,  but  any  zero  order  branch  of  a  first  order  branch 
still  be  called  a  1st  order  branch,  (e.  g.  ,  in  Figure  1  activities 
(2;  0,2),  and  (3;  0,2)  connecting  points  (0)  and  (2)  are  two 
second  order  branches  belonging  to  the  second  first  order 
branch.  Likewise  (7;  2,  5)  is  a  second  order  branch  belonging 
to  this  first  order  branch. 

2.11.  The  uncrossed  network  -  If  by  the  repeated  application  of 
definitions  2.]  to  2.10  all  individual  activities  in  the  network 
can  be  identified  as  different  k-th  order  branches  (for  some 

k  >0),  the  network  is  said  to  be  "uncrossed.  "  (e.  g.  ,  the  net¬ 
work  in  Figure  1  is  uncrossed  and  all  activities  are  recognized 
as  different  2nd  order  branches.  The  network  in  Figure  2  is 
likewise  uncrossed  with  some  of  the  individual  activities  being 
2nd  order  branches  and  some  3rd  order  branches.  However, 
the  network  in  Figure  3  is  crossed  -  there  being  only  one 
(0  order)  branch  comprising  all  activities. 
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3.  CROSSED  AND  MULTIPLE -CROSSED  NETWORKS,  The  arrange¬ 
ment  shown  in  Figure  3,  called  the  'Wheatstone  bridge1.  h*«  been  quoted 
in  tne  previous  section  as  an  example  of  a  crossed  network.  It  consists 
of  the  five  activities  (1;  0, 2),  (2;  0,1),  (3;  1,  2),  (4;  1,  3)  and  (5;  2,  3),  If 

now  each  of  these  five  single  activities  is  replaced  by  an  uncrossed  net¬ 
work,  as  defined  in  Section  2,  we  shall  reach  a  network  called  a  '1st  order 
crossed  network,  1  More  specifically  we  define  a  0-order  crossed  network 
as  an  uncrossed  network  in  which  at  least  one  of  the  'activities'  is  replaced 
by  a  Wheatstone  bridge  (see  Figure  3),  With  the  help  of  this  network  we 
define  a  t^-order  crossed  network  (for  t  ^  1)  as  a  0-order  crossed  net¬ 
work  in  which  any  'activity'  may  be  replaced  by  a  k^. order  crossed  net¬ 
work  with  0  <  k  <  t-1,  but  at  least  one  activity  is  replaced  by  a  (t-lj^-order 
crossed  network. 

Although  most  practical  situations  of  activity  networks  will  bs  recog¬ 
nised  as  a  t*h  order  crossed  network  for  some  order  t.  There  are  clearly 
quite  small  networks  which  do  not  belong  to  this  category,  as  for  example 
the  network  shown  in  Figure  4: 

4.  OPERATORS  FOR  EXACT  SOLUTION  BY  NUMERICAL  ANALYSIS. 
Consider  first  the  case  of  an  uncrossed  network  as  defined  in  sectioni. 

It  is  easy  to  show  (see  e.  g.  Section  5)  that  an  uncrossed  network  can  be 
built  up  from  individual  activities  by  two  basic  operations  which  can  be 
briefly  described  as  follows:  - 

Operation  it:  -  Placing  activities  in  parallel 

Operation  S:  -  Placing  activities  in  series 

These  basic  operations,  well  known  concepts  in  electric  circuit 
theory,  are  illustrated  in  Figures  5  and  6, 

Corresponding  to  these  two  basic  networks  we  now  develop  the  simple 
equation  for  the  c.  d.  f,  (cumulative  distribution  function)  of  the  'critical 
time1  in  the  two  basic  networks. 

a.  Parallel  activities:  - 

Denote  the  serial  number  of  the  k  activities  in  parallel  by  s  so  that 
•  =  1,  2,  .  ,  .  ,  k  (k  =  5  in  Figure  5)  and  denote  the  time  required  to 
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complete  the  s-th  activity  by  t  ,  If  the  c,  d,  f.  of  t^  by  F  (t  ) 

then  the  critical  time  t  for  thi*  eimple  network  is  clearly  given  by 

t=max  t  »o  that  the  c.  d.  f,  of  t  is  obtained  as 
s 

a 


(1) 


F(t)  =  Fr 


|  max  S  t 


•\ 

> 


k 

*  IT 

a=l 


Fa(t) 


b.  Two  activitie*  in  aeries. 


Denote  the  times  required  to  complete  the  two  activities  by  t  and 
t^  respectively  and  their  c .  d .  £.  '■  by  F^(t^)  and  F^t,,).  Then  the 
critical  time  for  this  simple  network  is  clearly  given  by  t  «  t^+t^ 
so  that  the  c.  d,  f,  of  t  is  obtained  as 


(2) 


pF  -1 

F(t)  -  \  F1(t-t2)dF2  ,  where  F  »  F2(t)  and  .  F2  (Fg), 

It  should  be  noted  that  equations  (1)  and  (2)  yield  the  c.  d.  f.  F(t) 
for  the  basic  network  from  the  c.  d,  f,  'a  of  the  individual  activities, 
Therefore,  these  basic  networks  can  subsequently  be  regarded  as 
'individual  activities'  and  entered  as  F  (t  )  in  subsequent  opera* 

tions  of  the  type  (1)  and  (2),  It  is  obvious  therefors  that  by 
repeated  application  of  (1)  and  (2)  the  c.  d,  f,  of  an  uncrossed 
network  such  as  in  Figure  1  and  Figure  2  can  be  obtained.  The 
operational  logic  for  this  is  given  in  section  5, 


Next  we  deal  with  1st  order  crossed  networks  and  to  this  end  must 
evaluate  the  c.  d,  f,  of  the  critical  time  t  for  the  Wheatstone  bridge  (figure 
3).  Denoting  by  t^,  ,  ,  ,  ,  t^  the  completion  times  for  the  five  activities 

s«l,  2, .  .  .  ,  5  as  arranged  in  Figure  3  and  by  F^(t^)  their  respective  c.  d,  f.  s 

we  obtain  by  elementary  probability  calculus  the  c ,  d .  f .  of  the  critical 
time  t  as  a  sum  of  three  integrals  as  shown  in  (3)  below:  • 
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(3) 


a  h  c 

F(t)  =  f  dF_  1  dF  f  dF  F  ft  +t  )  F  It  +t  ) 

j  e.  j  jj  o  i '  i  c  e  '  t  y 

a  d  f. 

+  \  dF2  j  dF4  \  dF  5  F1^2+t4  “V  F3^4  *  V 

+  ^  dFl  jl  dF5  y  dF2  F3  <V*2>  F4  <VV*2> 


-1 

where  t.  =  F.  (FJ  are  the  inverse  functions  of  F.(t,),  all  variables  of 
i  i  i  i  l 

Integration  are  the  F^  with  integrations  starting  at  ■  0  and  ending  at 
points  'a1  to  'J'  given  by 


a  ■  F2(t),  b  «  F 3(t -t2) ,  c  »  Fs(t-t2-t3) 


(4) 


d  -  F4(t-t2).  f  -  F5(t4),  g  -  Ft(t) 


h  "  F5^‘tl)’  J  "  r2^^‘ 


It  should  be  noted  that  the  three  terms  in  (3)  correspond  to  the  three 
mutually  exclusive  and  exhaustive  situations  (a),  (b),  (c)  shown  below 

(a)  Critical  path  t  *  +  *3  +  *3 

(b)  Critical  path  t  ■  t2  +  t4 

(c)  Critical  path  t  ■  t^  +  tj  , 

The  general  case  of  a  t-th  order  crossed  network  is  finally  covered 
by  repeated  application  of  the  above  operators  as  shown  in  section  5, 

5,  THE  COMPUTATIONAL  LOQIC  FOR  t-th  ORDER  CROSSED  NET 
WORKS.  The  computer  logic  shown  in  Figure  7  will  compute  the  c,  d.f.  of 
the  critical  time  in  a  t-th  order  crcesed  network  from  c.  d,  f.  s  of  the 
completion  times  of  the  individual  activities. 
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The  initialisation  of  the  computation  consists  of  loading  the  code  num- 
bo*«  ui  all  activities  (s;j,k)  (see  section  2,  3)  as  well  as  readying  the  tape 
giving  all  their  c.  d.  f,  functions,  If  the  serial  number  of  the  activity  is 
immaterial  we  shall  use  the  symbol  (.  ;  j,k).  In  the  course  of  the  operations 
•'ertain  code  numbers  will  be  deleted  and  the  retained  code  number  activities 
havb  thsir  c.  d.  f,  ’s  modified.  We  should  give  the  following  explanations 
of  so’,  .  of  the  operations  involved:  - 

Box  1;  2  An  activity  (a;  j,k)  with  the  current  serial  number  s  and  starting 

at  J  and  ending  at  k  (see  2,  3)  is  processed  i.  e.  s;  j  k  are  recorded 
and  the  aesor.iated  c,  d.  f.  F^(t)  loaded. 

Box  3  A  test  is  made  as  to  whether  there  is  a  2nd  activity  starting  at  j 
and  ending  at  k 

Box  4  If  the  2nd  activity  starting  at  j  and  ending  at  k  has  a  co<$«  (u;  j.k)  and 
ac.d.f.  of  r2(t),  replace  F^t)  by  F^(t)  F2(t)  and  delete  (uj  j,  k) 

from  the  list  of  code  numbers  and  F2(t)  from  the  tape  of  c.  d,  f, 

functions. 


Box  12  If  the  e,  d.  f,  functions  of  activities  (s;  J,k)  and  (•  ;k,n)  are  denoted 
by  F^t)  and  F ^(t)  respectively  we  replace  F^t)  by  T  F^t-t^  dF2 

-1  ° 

with  F  «  F2(t),  and  t2  ■  F’  (Fj,)  replace  the  code  (s;  j,  k)  by 
(s;j,n)  and  delete  code  (‘  ;k,n)  and  F.(t), 


Box  9  A  test  is  made  as  to  whether  the  current  activity  (si  j ,  k)  and 

associated  activities  (■ ;  j,  m),  (- ;  m,k),  (•  ;  m,n)  and  (■  ;  k,  n)  can 
be  identified  with  the  activities  (1;  0,  2),  (2:0,1),  (3;  1,2),  (4;  1,3) 
and  (5;  2,  3)  of  the  Wheatstone  bridge  of  Figure  3, 


Box  10  The  five  c.d.f.  functions  Involved  on  the  Wheatstone  bridge 
operation  are  combined  in  accordance  with  equation  (3).  The 
resulting  F(t)  replaces  F^(t),  the  code  (e;  j,n)  replaces  (s;  j ,  k) 
and  all  other  codes  and  cTd.f.  are  deleted. 


The  proof  that  the  logic  of  the  flow  diagram  in  Figure  7  does  indeed 
result  in  the  computation  of  the  c.d.f.  of  the  critical  time  for  any 
multiple-crossed  network  is  given  in  the  Appendix. 
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A  MQNTE  CARLO  SOLUTIONS  FOB.  THE  MORE  COMPLEX  NET¬ 
WORKS,  As  is  well  known  and  gs  was  mentioned  in  section  1  the  currently 
used  PERT  algorithm  determines  that  path  in  the  network  for  which  the 
total  of  average  completion  times  is  a  maximum.  Now  imagine  that  we 
apply  the  same  algorithm  to  a  random  sample  of  completion  times,  each 
drawn  from  the  distribution  relevant  to  its  activity.  The  'critical  time' 
so  computed  will  be  a  single  random  variable  from  the  distribution  of 
critical  times  defined  in  section  1  and  discussed  in  section  5.  A  large 
number  of  repetitions  of  this  computation  will  therefore  yield  a  Monte 
Carlo  solution  of  the  distribution  of  critical  times.  Such  a  solution  will 
therefore  be  available  for  any  network  (and  not  just  for  multiple  crossed 
networks) . 

Suppose  now  we  are  faced  with  a  complex  network  (not  necessarily 
multiple  crossed).  If  we  apply  the  algorithm  of  section  5  to  such  a  net¬ 
work  we  would  in  general  reduce  the  number  activity  -  codes  by  the 
operations  'tt',  'Conv'  and  'Bridge'.  However,  if  the  network  is  not 
multiple  crossed  we  shall  not  be  able  to  reduce  the  network  to  a  single 
activity.  As  soon  as  we  find  therefore  that  no  reduction  of  codes  has 
occurred  on  too  consecutive  cycles  we  would  output  the  reduced  network 
activities  and  associated  c.  d.  f,  '■  so  that  it  can  be  solved  by  Monte  Carlo 
as  indicated  above.  The  operational  calculus  of  section  5  will  considerably 
reduce  the  complexity  and  extent  of  the  network  so  that  the  subsequent 
Monte  Carlo  calculations  are  much  simplified. 

An  IBM  709  computer  program  performing  the  above  Monte  Carlo 
computations  of  the  distribution  of  critical  times  was  prepared  by 
L,  L.  McGowan  (1964),  in  his  M,  Sc.  thesis  at  the  Institute  of  Statistics 
at  Texas  A&iM  University. 

7,  SENSITIVITY  ANALYSIS  AND  GUIDE  TO  MANAGEMENT,  The 
previous  sections  have  been  concerned  primarily  with  the  establishment 
of  the  mathematical,  statistical,  and  logical  aspects  of  determining  the 
distribution  of  completion  times  for  a  project,  The  methods  developed 
have  further  applications  in  analysing  the  effects  of  making  specified 
changes  in  the  original  network  and  thereby  providing  guides  for  manage¬ 
ment  actions,  Basically,  the  analyses  most  readily  recognised  in  this 
area  are  concerned  with  (1)  assessing  the  impact  of  modifying  the 
distribution  of  specified  activities  (e,  g,  ,  a  change  in  their  average 
completion  times);  (2)  assessing  the  impact  of  modifying  blocks  of 
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activities;  (3)  comparing  two  or  more  network*  to  e*tabli*h  the  organise- 
tion  of  the  project  for  minimum  time,  minimum  co;t,  or  ;or»»e  other 
cptii.mrn;  and  (4)  assessing  progrei*  or  remaining  time  for  the  compie- 
tion  of  the  project. 


All  of  the  above  assessment*  are  permissible  under  the  method 
developed  in  this  paper.  In  fact  once  the  logic  is  established  on  a  com* 
puter,  all  four  assessments  are  possible  with  the  same  computer  programs 
It  is  only  necessary  to  vary  the  input  and  certain  problem  parameters 
according  to  the  assessment  required. 

It  should  be  pointed  out  that  the  afaessments  gained  via  this  logic  will 
be  more  comprehensive  than  a  similar  PERT  assessment.  With  the  pre* 
sent  logic  the  impact  on  the  c .  d .  £ .  of  project  completion  times  will  be 
observable.  This  means  that  our  sensitivity  analysis  provldts  estimates 
of  the  impact  of  production  schedule  changes  on  the  expected  completion 
time  but  also  of  the  impact  on  its  variance,  percentiles,  confidence  inter¬ 
vals  and  other  statlaticel  parameters. 

8.  SPECIAL  CASES  OF  BIAS  DEMONSTRATION.  As  noted  earlier 


bias  enters  the  solution  of  a  network  problem  due  to  inadequate  treatment 
of  the  statlaticel  considerations  and  approximate  logic,  In  order  to 
demonstrate  this  bias  a  few  examples  will  be  worthwhile  for  illustrative 
purpoeea.  The  following  examples  will  also  demonstrate  the  dependence 
of  the  solution  on  the  distribution  form  and  network  composition. 

EXAMPLE  1,  Consider  the  caae  where  k  activities  are  in  parallel  as  is 
illustrated  in  Figure  5.  Assume  further  that  each  t^  is  a 
random  variable  with  exponential  c,  d,  f. 


"Vi 

e  ,  i  ■  1,  2, 


t  £  0 


Thee,  d.f,  of  the  maximum  time  t  is  then  given 


F(t)  »  tt  (1  -  e  ) 
i«l 


If  X,  *  X  for  all  i,  the  mean  of  F(t)  is  given  by 
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(6) 


(7) 


EXAMPLE  2 


(8) 


Clearly,  since  all  X  =  X  and  hence  all  fi  ^  s  =  l/\  ,  the 

conventional  PERT  solution  under  this  condition  is  a  l/X  , 
The  bias  is  then  given  by 

k  i 

|i  -fiv  =  n*  E  — 

i  =  2  1 

Thus  if  there  are  only  k»4  activities  in  parallel  the  bias  will 

be  fjc  -fj,#  =  or  more  than  100  percent  of  the  PERT 

solution,  whilst  with  k=8  activities  in  parallel  the  bias  is 
1.  718  or  172%.  It  should  of  courss  be  remembered  that  the 
above  bias  applies  to  the  particular  network  in  Figure  5 
which,  in  general  would  only  constitute  a  small  section  of 
the  large  network.  Therefore,  the  %  bias  in  the  PERT  - 
computed  expected  completion  times  will  not,  in  general, 
be  as  large  as  the  above  example  would  indicate,  However, 
PERT  will  always  make  underestimates  of  the  critical  time 
intervals  (see  e,  g.  ,  Fulkerson  (1962),  p.  808)  so  that  the 
biases  from  individual  network  sections  will  cumulate. 

:  Consider  the  same  network  as  above  but  with  the  density 

functions  given  by  f(t, )  ■  L;  OS  t  i  c, 

1  c  1 

In  this  case 

F(t)  »  (t/c)k  ,  0  St  Sc  . 


The  mean  value  of  F(t)  is  then 
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F: 

VT- 


■ft-J 


The  PERT  solution  would  be  the  mean  value  of  tj  which  is 
j  .  The  bias  is  found  to  be 

do)  n  -  ^  =  ££  ^ 

In  this  example  the  bias  is  at  least  bounded  in  that  it  cannot 
exceed  100%  of  the  PERT  solution.  It  does  increase  very 
rapidly  however,  with  the  number  of  activities  in  parallel. 
If  k=4  as  in  the  first  example  the  biaa  is  60%  of  p.* ,  when 
k  =  8  it  is  78%. 

EXAMPLE  3.  To  illustrate  the  dependence  of  the  solution  upon  the  form 
of  the  densities  involved  consider  the  following  network. 


FIGURE  8 

SHIFT  OF  CRITICAL  PATH  WITH  FORM  OF 
DISTRIBUTION 


In  this  case  suppose  that  the  activities  represented  by  the 
t^  have  expected  times  as  follows;  - 
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If  conventional  PER?  is  applied,  path  ACE  will  be 
critical  with  a  sum  of  expected  times  of  17  units.  On 
the  other  hand,  if  the  densities  of  the  t^  are  exponential 

and  the  operational  logic  of  this  paper  is  applied  the 
expected  time  for  ABE  is  19  1/2  units,  for  ACE  17  units, 
and  ADE  19  units,  thus  making  ABE  critical.  This 
distribution  dependence  is  further  emphasized  if  the  t^ 

are  rectangularly  distributed.  In  such  a  case  the  expected 
time  for  ABE  is  16  1/2  units,  and  for  ADE  17  1/ 3  units, 
thus  making  ADE  critical. 

The  above  examples,  though  somewhat  elementary  and  academic, 
demonstrate  the  consequences  of  inadequate  statistical  treatment  and 
approximate  logic,  The  impact  can  be  even  more  pronounced  and  the 
consequences  more  significant  in  a  realistically  large  program  plan. 

9.  RELATION  TO  THE  EXISTING  LITERATURE  ON  PERT.  Most 
of  the  published  work  on  PERT  is  concerned  with  computations  based  on 
the  mean  values  of  the  completion  timeo  and  deliberately  ignores  the  bias 
discussed  in  this  paper.  There  are  undoubtedly  situations  when  this  bias 
is  not  serious  notably  in  networks  when 

(a)  There  is  a  low  degree  of  parallelism  in  the  activities  of  the 
network  and  most  operations  are  sequential  and/or 
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(b)  When  tome  activities  are  carried  out  <n  but  cr.c  cf  thorn 

has  a  considerably  longer  expected  completion  time  than  the 
others  parallel  to  it. 

It  will  be  agreed  that  the  above  conditions  are  not  usually  satisfied.  In 
view  of  the  very  extensive,  detailed  and  costly  computations  involved  in  the 
currently  practiced  PERT  analysis  it  is  surprising  that  So  little  attention 
has  been  paid  to  the  bias  affecting  them. 

We  believe  that  whilst  the  possibility  of  a  statistical  approach  (such  as 
is  here  presented)  has  sometimes  been  considered  (see  e.  g.  ,  Department 
of  the  Navy  (1958) ,  Appendix  A,  and  Fulkerson,  D.  R,  (1962))it  has  apparently 
been  regarded  as  leading  to  unsolvable  or  unmanageable  mathematics. 

Indeed,  Fulkereon  (1962)  who  fully  recognises  the  existence  of  the  bias 
(see  page  308)  and  offers  an  interesting  approximate  method  to  correct  it, 
states  (page  309):  -  "Sines  a  typical  PERT  network  may  Involve  hundreds 
and  thousanda  of  arcs,  the  precise  calculation  of  expected  critical  path 
lengths  would,  of  course,  be  out  of  the  question.  "  Now  it  must  of  course 
be  remembered  that  the  method  of  numerical  analysis  here  offered  gives 
the  eolutlon  only  for  the  special  caee  of  multiple-crossed  networks  as  here 
defined,  We  do  not  claim  that  the  networks  encountered  in  practice  will 
usually  belong  to  this  category,  However,  if  the  algorithm  described  in 
section  5  is  applied  to  a  general  network  it  will  reduce  it  considerably  so 
that  the  distribution  of  the  critical  time  for  the  reduced  network  can  be 
obtained  by  the  Monte  Carlo  procedure  described  in  section  6,  Moreover, 
we  could  enlarge  the  scope  of  the  numerical  method  of  section  5  by  adding 
(to  the  Wheatstone  bridge  operation  for  the  network  in  Figure  3)  similar 
basic  crossed  networks  (such  as  that  of  Figure  4)  and  incorporate  a  calcula- 
tion  of  the  critical  time  (aimilar  to  that  given  by  equation  (3))  for  euch 
configurations,  The  feasibility  and  economy  of  such  additions  is  under 
investigation, 

Since  we  only  give  a  hand  full  of  reference*  in  spite  of  the  vast 
literature  on  the  subject,  we  should  perhaps  include  the  extensive  Bibliog¬ 
raphy  (Bolling  Air  Force  Base  (1963))  in  our  list, 
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TEQUILAP:  TEN  QUANTITATIVE  ILLUSIONS 
OF  ADMINISTRATIVE  PRACTICE* 

Clifford  J.  Maloney 


In  1949  I  had  occasion  to  install  a  punched  card  tabulator  for  the  purpose 
of  machine  calculation  of  analyses  of  variance  arising  in  a  research,  develop¬ 
ment,  and  testing  program.  At  that  time  no  need  was  felt  to  impose  extra¬ 
ordinary  restrictions  on  the  procurement  and  utilisation  of  euch  equipment. 
About  the  time  of  the  beginning  of  the  Korean  War,  however,  higher  author¬ 
ity,  under  the  impression  apparently  that  punched  cards  were  employed 
only  in  Comptroller  functions  and  that  all  such  machinery  was  rented, 
instituted  a  requirement  for  monthly  reports  of  per  cant  utilisation,  with 
a  somewhat  informal  understanding  that  good  management  would  secure  a 
level  of  utilisation  of  each  piece  of  equipment  at  leaet  as  high  as  50ft  and 
that  80ft  would  be  much  more  appropriate.  Having  encountered  what  has 
since  come  to  be  called  queuing  theory  in  Thornton  Fry's  text  on  proba¬ 
bility  theory  many  years  earlier,  I  had  a  summer  worker  in  1955  make  an 
application  of  these  considerations  to  the  congestion  delays  Which  would 
result  from  any  given  level  of  per  cent  utilisation,  These  are  shown  in 
Figure  1,  This  study  has  appeared  in  a  paper  given  at  the  Second  Statis¬ 
tical  Engineering  Symposium  at  Edgewood  Arsenal  in  April  of  1956,  How¬ 
ever,  my  efforts  and  those  of  othere--a  few  of  which  have  come  to  my 
attention- -to  point  out  the  costs  as  well  as  the  benefits  as  per  cont 
utilisation  increases  had  an  absolutely  sero  effect  on  "administrative 
practice.  "  Two  major  conclusions  from  this  experience  and  many 
others,  before  and  since,  were,  however,  made  clear  to  me.  The  first 
conclusion  is  that  decision  making  is  an  emotional,  not  a  rational,  opera¬ 
tion,  People  often  bolster  their  decisions  by  arguments- -  some  of  them 
rational--but  seldom  reverse  the  process,  I  am  sorry  to  say  that  so  far 
as  I  can  see  this  holds  as  much  for  logicians  and  mathematicians  as  for 
anyone  else,  This  is  of  course  what  is  meant  by  that  well  known  eaylng; 

"I've  already  made  up  my  mind;  don't  bother  me  with  the  facts,  "  The 
second  conclusion  deals  with  the  arguments  by  which  it  is  customary  to 
rationalize  emotional  decisions.  Even  where  the  "reasoning"  can  be 
accepted  as  not  totally  irrelevant,  it  will  be  based,  not  invariably  but 
very  often,  on  unwarranted  but  unquestioned  assumptions.  My  example 


* T he  views  expressed  herein  are  those  of  the  author  and  are  not  to  be 
ascribed  to  any  other  agency  or  individual, 
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of  per  cent  utilization  aa  a  measure  of  punched  card  installation  efficiency 
illustrates  this.  One  assumption  was  that  the  equipment  was  rented  and 
not  owned.  Another,  that  efficiency  was  not  merely  •  lum-ilor.  of  utilize, 
tlon  but  actually  a  monotone  function  of  it. 

1.  cannot  claim  originality  in  this  insight,  In  a  commencement  address 
at  Yale  University,  June  11,  1962,  the  President  of  the  United  States  said: 
"For  the  great  enemy  of  the  truth  is  very  often  not  th*  lie --deliberate 
contrived,  and  dishonest,  but  the  myth- -presistjnt,  persuasive,  and 
unrealistic,  Too  often  we  hold  fast  to  the  cliches  of  our  forebears,  We 
subject  all  facts  to  a  prefabricated  set  of  interpretations,  We  enjoy  the 
comfort  of  opinion  without  the  discomfort  of  thought  .  ,  ,  ,  Mythology 
distracts  us  everywhe  re  - -in  government  as  in  business,  in  politics  as  in 
economics,  in  foreign  affairs  as  in  domeetic  policy.  "  The  former 
President's  indictment  is  much  stronger  and  more  inclusive  than  mine, 

Allyn  Kimball*  has  defined  "errors  of  the  third  kind"  as  giving  correct 
answers  to  the  wrong  questions.  The  assumptions  of  the  question  become 
postulates  of  the  answer,  He  obeervee  that  a  first  step  in  finding  useful 
answere  is  to  query  the  question.  The  originator  of  the  cognate  Insight 
that  most  of  what  in  common  life  passes  for  argument  consists  of  more  or 
less  accurate  deduction  from  wrong  premises  is  lost  in  the  milts  of  time, 
But  perhaps  there  is  room  for  me  to  aietrt  some  email  claim  to  origi¬ 
nality  in  the  recognition  that  many  of  these  false  premies e  aprlng  from  an 
inability  or  an  unwillingneee  to  think  in  quantitative  terms,  to  see  the 
clarifying  role  of  an  appreciation  of  their  quantitative  nature,  and  to  see 
that  some  at  least  of  these  false  deductions  cannot  be  reeolved  by  logic 
alonei  as  their  very  nature  is  inherently  quantitative, 

The  application  of  mathematical  principles  in  administrative  practice 
which  I  illustrated  in  my  first  example,  specifically  an  application  of  prob¬ 
ability,  le  relatively  sophisticated,  Further,  the  problem  which  gave  rise 
to  it,  while  important,  was  rather  limited  in  scope.  Moet  of  what  adminis¬ 
trators  do  day  in  and  day  out  consists  of  more  homely  if  more  important 
actions,  though  it  is  true  that  queuing  theory  has  many  applications  there 


*  Allyn  Kimball,  "Errors  of  the  Third  Kind  in  Statistical  Consulting,  " 
JOURNAL  OF  THE  AMERICAN  STATISTICAL  ASSOCIATION,  June  1957, 
p,  133. 
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also.  Dr.  Edward  F,  R.  Hearle*  in  an  article  "How  Useful  Are  'Scientific' 
Tools  of  Management?"  enumerates  them  as:  "linear  and  dynamic  program¬ 
ming,  queuing  theory,  game  theory,  simulation,  and  monte  carlo,  to  name 
a  few.  "  The  general  tenor  of  his  appreciation  of  these  tools  and,  hence, 
of  quantitative  thinking  in  management  is  believed  expressed  in  his  sentence: 
"Furthermore  these  tools  do  not  deal  with  some  of  the  more  exciting  parts 

of  the  total  management  process . "I  take  exactly  the  opposite  view 

to  the  one  that  I  believe  Dr.  Hearle  is  espousing;  that  quantitative  thinking 
is  not  (as  he  asserts)  limited  to  formal  manipulation  of  numerical  quantities 
but  is  very  useful  where  the  mere  recognition  of  a  variation  from  instance 
to  instance  of  a  given  type  is  involved.  *  * 

The  serviceability  of  adhering  rigidly  to  the  "channels"  of  an  orginiza- 
tion  chart  can  be  judged  by  indicating  the  lines  of  contact  that  exist  in  the 
absence  of  "organization.  "  (Figure  2).  The  organization  chart  of  Figure  3 
was  selected,  not  to  suggest  that  administration  goes  around  in  circles,  but 
because  the  chart  is  round  and,  hence,  emphasizes  the  contrast  with  Figure 
2.  The  organization  replaces  an  unorganized  conglomerate.  Now,  when 
the  production  department  gets  ready  to  fill  an  order,  they  send  the  goods 
and  the  invoice  to  the  President's  office,  and  his  secretary  passes  them 
on  to  the  shipping  and  accounting  departments.  At  least  no  organization 
chart  of  which  I  am  aware  gives  any  guidance  to  the  contrary.  This  may 
be  the  reason  that  organization  charts  and  the  "authority  lattices"  behind 
them  have  such  a  low  reputation  outside  of  organization  and  management 
departments.  There  is  a  great  deal  of  discussion  and  some  useful 
research  which  distinguishes  between  "formal"  and  "informal"  organiza¬ 
tional  relationships,  but  a  recognition  that  a  great  gulf  so  frequently  exists 


*Dr.  Edward  F.  R.  Hearle,  "How  Useful  Are  'Scientific'  Tools  of 
Management?'  PUBLIC  ADMINISTRATION  REVIEW,  Autumn  1961,  pp.  206- 
209. 

**In  making  his  statement  Dr.  Hearle  had  the  type  of  mathematical  tools 
which  he  had  enumerated,  and  others  like  them,  in  mind.  So  my  challenge 
to  him  relates  to  the  inference  which  can  fairly  be  drawn  from  his  remarks 
rather  than  to  any  direct  statement  which  he  makes. 
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between  these  structures  is  itself  a.  witness  of  an  inadequate  formal  structure 
in  the  organisation- -as  is  the  fact  that  tl  j  disparity  is  so  widespread  a 
testimony  to  the  unrealistic  (and  unhelpful  nature  of  current  theories  of 
organization. 

The  comparatively  sterile  status  of  organisation  theory  may  be  due 
to  the  inherent  difficulty  of  the  subject,  the  low  ability  of  those  who  work 
in  the  field,  or  to  the  absence  of  one  or  a  few  essential  concepts  not  yet 
sufficiently  clearly  delineated.  The  author  is  encouraged  to  propose  that 
the  latter  may  be  an  essential  feature,  and  that  the  principles  to  be 
discussed  may  be  included  among  the  missing  essential  elements.  Tew  of 
these  points  are  entirely  original,  yet  essentially  none  are  clearly  and 
widely  appreciated. 

Reflection  of  many  years,  exponentially  enhanced  in  intensity  in 
recent  months,  has  led  ms  to  subsume  the  most  significant  examples  of 
quantitative  illusions  in  administrative  practice  which  have  c.oma  to  my 
attention  under  10  headings,  partly  because  I  have  10  fingers,  and  partly 
because  this  produced  the  aeronymlc  title.  This  paper  will  consist  of  a 
listing  of  these  10  principles  with  a  few  exemplifications. -not  to  convince 
anyone  of  the  truth  of  my  position  on  ths  examples:  it  would  be  impossible 
to  discuss  mors  than  on#  or  two  in  ths  little  space  allotted  to  m*«»but  to 
demonstrate  the  clarification  1st  into  many  administrative  forms  of  action 
which  otherwise  must  remain,  as  they  were  formerly  to  ms  and  must  still 
bs  to  you,  an  enigmatic  mystery  If  ths  reader  feel*  that  my  examples  are 
inadequate  or  wrong,  he  ie  invited  to  refer  to  others  from  his  own  expe> 
risnce,  Only  if  he  feels  that  none  or  few  can  be  found  does  he  have  a 
quarrel  with  my  principles  psr  se, 

My  first  "quantitativs  illusion  of  administrative  practice,  "  (Figure  4), 
is  entitled  "Peat  in  a  Pod,  "  to  emphasise  that  (a)  the  assumption  that  1s 
gtnsrally  mads  is  that  "if  ths  nims'i  ths  sams,  ths  thing's  ths  sams"-- 
"as  alike  as  peas  in  a  pod,  "  and  (b)  that  the  assumption  is  wrong.  1  see 
this  as  ths  exact  opposite  of  Profeieor  Hearle's  (implied)  position  cited 
earlier.  We  can  gain  greatly  by  merely  recognising  this  fallacy,  without 
in  any  way  being  able  to  quantify  it.  This  is  at  ones  the  most  important 
and  the  moat  fundamental  of  my  10  illusions.  The  fallacy  lias  in  the 
denial  of  the  reality  and  significance  if  quantitation  in  situation!  where 
it  is  real  and  important,  Sometimes  it  takca  the  elightly  more  subtle 
form  of  acknowledging!  yea,  the  several  member*  of  any  ona  elan  do 


Design  of  Experiments 


413 


differ  and  perhaps  differ  widely,  but  it  is  administratively  impossible  to 
allow  for  every  possible  variation;  therefore,  we  will  allow  for  none, 

The  same  punched  card  computing  installation  provided  a  glaring  example 
of  this  "reasoning.  11  We  needed  a  card  punch  operator,  The  position 
analyst  referred  to  a  job  standard  which  explained  that  card  punch  oper¬ 
ators  work  from  edited  repetitions  item  records  which  are  punched 
mechanically  with  no  exercise  of  individual  intelligence  or  ingenuity. 

The  position  analyst  had  no  need  to  look  at  the  facts  when  he  had  a  book 
to  tell  him  that  (a)  this  position  was  identical  with  all  other  positions 
called  by  the  same  title,  and  (b)  one  such  position  filled  the  book 
specifications,  Another  exemplification  of  the  illusion  which  receives 
a  great  deal  of  public  attention  is  lowest  bid  procurement,  Since  all 
items  (even  those  not  yet  invented)  and  all  services  can  be  exactly 
described  in  the  invitation  to  bid,  all  are  equivalent;  and  hence  the  pur¬ 
chase  price  is  the  one  remaining  variable.  Of  course,  there  is  a 
contrary  cliche;  "you  get  what  you  pay  for.  "  There  is  no  requirement 
in  "administrative  practice"  that  the  system  of  cliches  be  consistent,  * 

My  second  illusion  (Figure  5)  attempts  to  deal  with  the  assumption 
that  all  the  good  qualities  reside  in  one  product  or  one  course  of  action, 
and,  by  inescapable  logic,  all  the  bad  lie  in  any  alternative- -though  if 
there  are  several,  these  bad  qualities  may  be  distributed  among  them, 

I  am  sorry  to  say  that  I  was  supplied  with  a  perfect  example  of  this 


•  It  is  a  truly  remarkable  thing  that  philosophers,  since  the  time  of 
Plato,  have  been  concerned  with  the  problem  of  '’nominalism"  versus 
"realism,  "  which,  however,  important  theoretically,  seems  not  to 
constitute  a  stumbling  block  in  day  to  day  relationships;  whsreas  this 
first  illusion  is  at  once  the  most  pervasive  and  most  pernicious  logical 
fallacy  entering  not  just  into  almost  every  discussion  betwsen  friend 
and  foe,  between  advocate  and  adversary,  but  between  even  so 
intimately  related  and  favorably  disposed  groups  as  members  of  ons 
family,  It  was  the  essence  of  the  "hyphenated-Amcrican"  dispute,  the 
merits  and  the  abuses  of  political  party  labels  and  party  loyalties, 
methods  versus  subject  matter  in  education  and  of  occupational  juris¬ 
dictional  disputes,  whether  within  one  organisation  or  between  compet¬ 
ing  parties  or  groups. 
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*UB*d  by  special  peialoalcm  of  KING  FEATURES  SYNDICATE, 
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illusion  in  the  16  October  1964  issue  of  SCIENCE.  Two  of  this  country's 
most  illustrious  scientists  explained  their  choice  of  candidate  for 
President,  I  have  examined  these  statements  with  some  care.  Neither 
protagonist  could  find  a  fault  worth  mentioning  in  his  own  choice  nor  a 
good  quality  in  the  latter's  opponent.  This  action  constitutes  a  conform¬ 
ance  to  the  practice  in  disputation.  *  But  it  is  the  fact  that  the  practice 
prevails  in  the  day  to  day  administrative  process  that  concerns  us. 

Why  does  it?  The  complete  explanation  must  lie  in  psychology.  A 
plausible  treatment  of  just  this  phenomenon  has  been  contributed  by 
Professor  Leon  Festinger  of  Stanford  University.**  Of  course,  Profes¬ 
sor  Festinger  is  not  responsible  for  my  understanding  or  use  of  his 
theory.  In  essence,  the  mind  demands  harmony.  Yet  all  real  things 
and  all  real  courses  of  action  involve  advantages  and  disadvantages. 

The  only  achievable  harmony  is  a  quantitative  one --a  balancing  of 
opposing  forces.  But  this  type  of  harmony  is  uncongenial  to  many 
minds. 

That,  if  one  embraces  one  course  of  action  or  one  belief,  he  must 
impute  all  virture  to  the  chosen,  and  all  evil  to  the  rejected,  was  long 
ago  recognized  as  a  fundamental  error  by  Georg  Wilhelm  Hegel,  last 
of  the  global  philosophers,  who  saw  progress  of  social  organization  in 
the  reconciliation  of  the  thesis  and  the  antithesis  into  a  synthesis  that 
removed  the  conflict  by  absorption  of  the  thesis  and  antithesis  as 
elements  in  a  higher  concept.  Why  Hegel  never  "caught  on"  in  adminis¬ 
trative  practice  I  cannot  say.  But  it  is  possible  that,  since  his  view 
was  essentially  qualitative  and  not  quantitative,  hence  didn't  in  fact 
apply  in  many  instances,  his  solution  tended  to  be  neglected  even  when 
perfectly  applicable.  An  entirely  analogous  situation  is  known  to  have 
delayed  acceptance  of  the  contagion  theory  of  disease  for  centuries. 


*One  of  the  authors  was  good  enough  to  acknowledge  receipt  of  an  early 
draft  of  this  paper  with  the  statement  that  the  allocation  between  the 
two  discussions  was  a  deliberate  attempt  to  conform  to  the  anticipated 
expectations  of  the  readership  of  SCIENCE.  I  do  not  have  the  boldness 
to  point  out  that  insofar  as  the  anticipation  is  correct,  my  strictures 
apply  then  to  the  readership  if  not  the  disputants. 

**Leon  Festinger,  "A  Theory  of  Cognitive  Dissonance,  "  (Evanston, 
Illinois,  Row,  Peterson,  1957)  pp.  260-285. 
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Those  who  escape  the  first  illusion  and  recognize  that  quantitative 
variation  pervades  most,  if  not  all,  of  life  are  candidates  for  the  notion 
that,  if  a  little  is  good,  more  is  better.  (Figure  6),  The  "spartan" 
philosopher  would  put  that  from  the  standpoint  that,  if  moderation  is 
good,  abstinence  is  ideal.  1  have  seen  this  illusion  active  in  the 
question  of  where  to  put  the  statistician  in  an  organization.  The  same 
holds  for  engineers,  stenographers,  air  support,  artillery,  computers 
and  machine  tools.  If  the  drawbacks  of  the  widest  possible  organize* 
tional  scattering  are  recognized,  then  complete  centralization  appears 
ine scapable - -and  vice  versa  to  most  administrators,  organization 
specialists,  operations  analysts  and  other  people  who  move  productive 
workers  around,  The  most  extreme  proposal  that  has  come  to  my 
attention  is  to  centralize  all  computers  in  government.  The  one  facet 
of  all  these  matters  that  is  of  concern  here  ie  the  human  tendency, 
once  embarked  on  a  path,  to  assume  that  that  path  leads  upwards  (or 
downwards)  indefinitely.  The  visible  existence  oi  "side  effects"  in 
therapeutics  has  compelled  an  avoidance  of  this  illusion  with  full 
virulence  in  medical  practice  - -but ,  as  recent  history  show*,  the 
tendency  certainly  exists. 

The  recognition  of  the  existence  - -though  1  think  not  the  nature-- 
of  this  tendency  has  led  those  who  prefer  the  statue  quo  to  take  refuge 
in  warnings  against  "the  foot  in  the  door,  "  "the  opening  wedge,  "  "the 
breach  in  the  dike,  "  "the  camel's  note  under  the  tent,  "  with  the 
consequence  that  anyone  who  feare  extremism  acte  like  and  ie  char¬ 
acterized  ae  an  obstructionist.  A  recognition  that  the  action  or  state 
rebelled  againat  would  lose  Its  terrors  (so  that  its  advantages  could 
be  secured)  if  illustion  three  could  be  eliminated  might  remove  much 
acrimonious  social  debate. 

Forty  years  ago  there  was  a  principle  in  psychology,  that  I 
have  never  heard  mentioned  since,  which  involves  a  modified  form 
of  this  fallacy  and  provides  a  partial  sxplanation  for  its  existence, 

1  can  best  exhibit  its  nature  by  recalling  to  mind  the  ones  popular 
medical  treatment  of  bleeding.  General  Washington  was  bled  in  his 
last  illness  and  the  practice  continued,  though  with  declining  popular¬ 
ity,  to  the  Civil  War,  and  even  lingered  till  World  War  I,  Suppose  a 
patient  is  "treated"  by  bleeding  but  succumbs.  Three  conclusions 
are  possible.  The  bleeding  either  was  (1)  deleterious,  (2)  indifferent, 
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Those  who  escape  the  first  illusion  and  recognise  that  quantitative 
variation  pervade*  most,  if  not  all,  of  life  are  candidate*  fnr  th*  notion 
that,  if  a  little  is  good,  more  is  better.  (Figure  6),  The  "spartan" 
philosopher  would  put  that  from  the  standpoint  that,  if  moderation  i* 
good,  abstinence  is  ideal.  I  have  teen  this  illusion  active  in  the 
question  of  where  to  put  the  statistician  in  an  organization,  The  same 
holds  for  engineers,  stenographers,  air  support,  artillery,  computers 
and  machine  tools.  If  the  drawbacks  of  the  widest  possible  organisa¬ 
tional  scattering  are  recognized,  then  complete  centralization  appears 
inescapable--and  vice  versa  to  most  administrators,  organization 
specialists,  operations  analysts  and  other  people  who  move  productive 
workers  around,  The  most  extreme  proposal  that  has  come  to  my 
attention  is  to  centralize  all  computers  in  government,  The  one  facet 
of  all  these  matters  that  Is  of  concern  here  is  ths  human  tendency, 
once  embarked  on  a  path,  to  assume  that  that  path  leads  upwards  (or 
downwards)  indefinitely,  The  visible  existence  of  "side  effects"  in 
therapeutics  has  compelled  an  avoidance  of  thie  Illusion  with  full 
virulence  in  medical  practice- -but,  as  recent  history  shows,  the 
tendency  certainly  exists. 

The  recognition  of  the  existence --though  I  think  not  the  nature «■ 
of  thie  tendency  has  led  those  who  prefer  the  statue  quo  to  take  refuge 
in  warnings  against  "the  foot  in  the  door,  "  "the  opening  wedge,  "  "the 
breach  In  the  dike,  "  "the  camel's  nose  undsr  the  tent,  "  with  the 
consequence  that  anyone  who  fears  extremism  acts  Ilka  and  is  char¬ 
acterized  as  an  obstructionist,  A  recognition  that  the  action  or  stats 
rebelled  against  would  lose  its  terrors  (so  that  its  advantages  could 
be  secured)  if  illuition  three  could  be  eliminated  might  remove  much 
acrimonious  social  debate, 

Forty  years  ago  there  was  a  principle  in  psychology,  that  I 
have  never  heard  mentioned  since,  which  involves  a  modified  form 
of  this  fallacy  and  provides  a  partial  explanation  for  its  existence, 

I  can  best  exhibit  its  nature  by  recalling  to  mind  the  once  popular 
medical  treatment  of  bleeding,  General  Washington  wai  bled  in  his 
last  illness  and  the  practice  continued,  though  with  declining  popular¬ 
ity,  to  the  Civil  War,  and  even  lingered  till  World  War  I.  Suppose  a 
patient  Is  "treated"  by  bleeding  but  succumbs.  Three  conclusions 
are  possible.  The  bleeding  either  was  (1)  deleterious,  (2)  indifferent, 


SO  MUCH  THE  BETTER 
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or  (3)  helped  but  was  inadequate.  In  the  third  case,  the  remedy  is  even 
more  heroic  bleeding,  This  latter  conclusion  is  associated  most 
strongly  with  the  name  of  Dr,  Benjamin  Rush,  Professor  in  the  first 
medical  school  in  America, 

In  the  psychological  context  this  proce ss- -the  persistence  in  a 
wrong  course  of  action  under  the  misapprehension,  entirely  sincere, 
that,  while  the  strength  was  weak,  the  sense  was  right- -was  called 
the  beta  hypothesis.  It  seems  most  curious  that  such  a  fecund  insight 
should  drop  from  view, 

But  that  it  is  more  important  to  run  in  the  right  direction  than  to 
gain  great  yardage  was  redsmonstrated  in  a  football  game  Sunday, 

25  October  1964,  by  Minnesota  end  Jim  Marshall.  A  gyroscope  persists 
on  the  course  of  Its  setting  despite  contrary  forces,  But  does  that  make 
the  setting  right?  Methods  for  determining  the  "direction  of  choice" 
have  long  been  known  in  the  science  of  statics  and  have,  more  recently, 
been  investigated  in  economics,  The  glaring  fact  that  scholars  cannot 
agree  on  the  direction  of  the  consequences  of  an  economic  action  is  a 
greater  blow  to  the  status  of  the  science  of  economics  than  the  uncer¬ 
tainty  of  the  magnitude  of  the  effect  in  those  few  cases  where  there  is 
agreement  on  the  direction, 

Innovator*  since  the  beginning  of  time  have  regarded  all  who  saw 
merit  in  the  old  ae  obstructionist* ,  That  they  may  merely  be  victims 
of  the  beta  virus  has,  ao  far  aa  I  know,  never  been  entertained, 
Galileo's  experience  is  the  moat  famous,  if  not  the  most  meritorious, 
examplt,  but  ths  theme  is  commonplace.  It  is  a  reliable  story  to  till 
in  the  movies,  on  TV,  and  in  ths  pulp  prsss,  In  real  life  it  ie  ae  com¬ 
mon  in  criminal  prosecution,  apparently,  as  in  scientific  innovation. 

In  the  common  view,  stubborn  insistsnee  on  ons  view  and  a 
refusal  to  even  hear  the  evidence  for  another  is  a  sign  of  malevolence 
on  the  part  of  the  defenders  of  the  status  quo,  In  ths  psychological  beta 
hypothesis  it  was  a  particular  form  of  mental  illness,  possibly  mild 
in  character  and  little  disabling,  In  a  seemingly  neglected  paper  in 
SCIENCE,  the  philosopher,  Michael  Polanyi,  citing  his  own  severe 
injury  from  exactly  this  proceae  nevertheless  defends  it  as  essential 
to  progress  in  science.  He  says;  "there  must  be  at  all  times  a 
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predominantly  accepted  scientific  view  of  the  nature  of  things  in  the  light 
of  which  research  is  jointly  conducted  by  members  of  the  community  of 
scientists.  A  strong  presumption  that  any  evidence  which  contradicts 
this  view  is  invalid  must  prevail.  Such  evidence  has  to  be  disregarded 
even  if  it  cannot  be  accounted  for,  in  the  hope  that  it  will  eventually  turn 
out  to  be  false  or  irrelevant.  " 

This  thesis  of  Professor  Polanyi  seems  to  be  receiving  just  the 
treatment  which  he  argues  it  must.  Despite  his  evident  eminence,  both 
as  scientist  and  as  philosopher,  he  appears  to  be  suffering  from  an  acute 
case  of  the  second  (Profit  and  Loss)  illusion,  complicated  by  the  presence 
of  the  beta  virus.  He  sees  that  science,  functioning  as  it  does,  advances. 
He  does  not  see  how  it  could  do  so  were  it  to  purify  itself. 

It  is  clear  that  the  seriousness  of  the  third  illusion  is  unmistakable, 
once  its  reality  is  granted,  but  when  combined  with  the  second  illusion 
(that  one  of  alternative  courses  of  action,  degrees  of  centralization,  size 
of  computer,  has  all  the  virtures  and  none  of  the  faults --or  that  one 
must  at  least  act  that  way)  explains,  I  believe,  the  peculiar  property  of 
progress  in  administration.  (Figure  7)  At  one  extreme,  (form  of 
organization,  method  of  scheduling  work,  approval  channels,  work  pro¬ 
cedure,  etc.  )  the  adherents  of  the  other  will  have  ample  proof  of  the 
inadequacy  of  the  chosen  solution- -and  they  will  be  right.  Fears  of  the 
"opening  wedge"  effect  will,  however,  maintain  the  existing  status  quo 
until  proponents  of  the  opposite  extreme  gain  sufficient  ascendency  to 
overcome  these  fears,  when  a  shift  towards  a  balanced  solution  will  set 
in.  Now,  just  when  the  optimum  position  is  reached,  the  third  illusion 
will  add  momentum  to  the  swing  until  the  opposite  extreme  is  attained. 

In  consequence  of  the  interaction  of  these  two  illusions,  "Profit  and 
Loss"  and  "So  Much  the  Better,  "  progress  in  administrative  practice-- 
and  many  other  facets  of  human  behavior--consists  in  discarding  yester¬ 
day's  procedure  and  adopting  that  of  the  day  before,  with  the  assumption 
and,  indeed,  the  claim  that  the  new  is  novel.  There  is  no  risk;  for  few 
will  have  survived  from  the  earlier  period,  and  they  can  be  suppressed. 
The  number  who,  not  personally  surviving,  will  have  read  history  will 
be  of  measure  zero. 
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Prsfccso;  Jiy  V*',  Forrester  oi  M.i.  l.  arguee*  that  thi*  result  it 
in  fact  a  necessary  and  not  accidental  characteristic  of  administrative 
progress  under  the  existing  circumstances.  He  writes;  "In  the  past, 
management  methods  have  been  learned  primarily  through  personal 
experience.  The  developing  manager  rotates  through  numerous  assign- 
ments,  Management  schools  repeat  the  folklore  and  the  experiences 
of  practicing  managers.  This  experience  is  used  as  a  basis  for  generali¬ 
zing,  so  that  past  experiences  can  become  a  basis  for  anticipating  the 
nature  of  new  situations.  " 

",  .  .  we  have  here  at  the  M.I,  T.  School  of  Industrial  Management 
been  developing  an  approach  to  management  policy  design  which  we  ctll 
'industrial  dynamics,  '  It  is  intended  to  be  a  new  way  to  understand  how 
corporate  structure  and  policy  produce  the  different  characteristics 
which  one  sees  in  business  enterprises,  ,  ,  ,  Mott  managers  are 
surprises  to  learn  that  those  practices  which  thsyknow  thsy  are  follow¬ 
ing  are  sufficient,  when  assembled  in  a  system  model,  to  cause  the 
major  difficulties  which  thsy  have  been  experiencing.  " 

Professor  Forrestsr  is  too  polits  to  do  so,  but  1  cannot  rssist 
observing  that  the  j  ibe:  "He  has  never  met  a  payroll"  is  a  pistol  pointed 
backward. 

Figure  S  shows  another  illusion  widsly  prevalent  but  not  recognised 
as  quantitative,  It  is  clossly  akin  to  illusion  one,  but  differs  in  that 
here  the  underlying  phenomena  is  recognised  as  quantitative,  s,  g.  , 
different  grads  levels  for  different  duties,  adjustments  in  the  general 
level  of  compensation,  fringe  benefits,  quality  of  tools  or  equipment, 
work  space,  and  so  on,  but  It  is  assumed  that  all  maxima  are  cusps; 
that  if  some  precise  (even  if  unknown)  value  is  optimum,  then  that  even 
slight  departures  result  in  great  waste  of  resources.  My  own  chief 
experience  in  this  field  had  to  do  with  the  erection  of  a  building  to  house 
a  computer  and  associated  staff,  W»  were  asked  to  build  for  the  future, 
but  to  justify  in  great  detail  just  how  every  square  foot  would  be  assigned, 


“Jay  W,  Forrester,  "Dynamics  of  Corporate  Growth,  "  Paper  delivered 
at  a  conference  on  "Management  Strategy  for  Corporate  Growth  in  New 
England,  "  held  at  M,  I.  T,  November  12,  1963,  This  brilliant  paper 
deeervts  reading  in  its  entirety  for  its  positive  approach  to  rescuing 
administrative  practice  from  the  grip  of  injurious  if  plausible  "folklore,  " 
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The  lntereat*  of  the  government  would  be  served  if  we  were  adequately 
housed,  but,  though  the  building  would  have  a  life  expectancy  measured 
in  decades,  it  seemed  to  be  assumed  that  any  attention  to  contingencies 
beyond  the  needs  of  the  instant,  (e,  g.  ,  an  anticipation  of  the  tortuous 
delays  in  the  path  to  actuality)  whether  of  our  unit  or  of  other  units  on 
the  Post,  could  only  be  "wasteful.  "  Opposition  to  grade  escalation;  to 
better  than  "necessary"  research  facilities,  military  aircraft,  weapons, 
uniform  belt  buckles,  or  missile  boosters  fall  in  this  category.  If  I 
say  that  I  think  this  line  of  "thought"  led  to  the  Russian  scoop  in  space, 
the  importance  of  this  illusion,  if  not  its  actuality,  will  be  obvious. 

In  his  treatment  of  "The  Economics  of  American  Medicine"  Seymour 
E.  Harris,*  it  seems  to  me,  suffers  severely  from  this  illusion  when 
he  refers  to  the  "waste"  of  better  than  "necessary"  medical  attention  or 
hospitalisation. 

Illusion  five  (Figure  9)  is  more  conventional.  At  first  blush  one  would 
assume  that  the  factor  which  varies  between  the  three  figures  is  one  of 
height  only,  despite  the  fact  that  we  all  know  that  the  human  figure  is  a 
three-dimensional  object;  hips,  waist,  and  of  course,  height,  Perhaps 
others.  I  include  this  illusion  only  to  pay  homage  to  a  factor  well 
recognised  in  all  circles,  mathematical  and  anti -mathematic^,  the 
multi -dimensionality  of  real  life  problems,  The  reader  is  asked  to 
stretch  a  point  in  the  Interest  of  economy  of  illustration  and  view  the 
figures  as  sdso  suggesting  the  non-linear  (curvilinear)  character  of  most 
realistic  situations.  Again,  the  availability  of  the  computer  could  push 
these  difficulties  back  a  step  or  two,  ware  the  nonquantltatively  oriented 
administrator  aware  of  the  potentialities, 

This  illusion  is  unique  in  that  it  occurs  with  great  frequency  in  two 
opposite  forms.  The  tendency  to  "oversimplify  a  problem"  is  widely 
indulged- -but  widely  condemned.  Indeed  the  opposition  to  automation, 
to  "thinking  by  computer,  "  while  reaching  a  crescendo  in  modern  times, 
has  had  its  Cassandra*  in  ail  ages. 


*Seymour  E,  Harris,  "The  Economic*  of  American  Medicine  "  (New 
York,  Macmillan,  1964). 
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Tlicie  anti-mathematicians  assert  tnat  no  amount  ol  complicating 
the  model  can  hope  to  mirror  reality  and  that  the  administrator  deals 
with  reality,  There  is  a  complementary  consequence,  The  adminis¬ 
trator,  like  the  observer,  "never  knows  what  he  is  talking  about  or 
whether  what  he  conclude 3  is  true,  "  and  without  recognising  it,  he  has 
conceded  as  much  by  the  above  argument.  Philosophers  of  the  scien¬ 
tific  method  have  repeatedly  pointed  out  the  contrast  between  the  rigor 
of  conclusions  derived  from  contrived  but  designed  experiments  and 
from  passive  observation  of  a  complex,  if  real,  world. 

A  particularly  striking  example  of  this  phenomenon  appeared  in  a 
recent  column  of  Walter  Lippmann,  concerning  the  place  in  history 
of  Herbert  Hoover.  Lippmann  says;  "we  avoided  such  a  crash  [ths 
Great  Depression]  after  the  Second  World  War  because  we  had  so  wall 
lsarned  the  lessions  of  the  First  World  War.  "  Clearly,  we  didn't  learn 
the  larger  lesson  of  how  to  avoid  war.  And  is  our  continuing  prosperity 
a  reward  for  a  lesson  well  learned  or  a  penalty  for  a  cold  war  still  in 
progress  ? 

A  widely  publicised  illustration  of  this  phenomena  hae  arisen  in 
the  controversy  over  emoking  and  disease.  The  evidence  in  the  human 
is  observational  and  not  experimental.  The  same  administrators  who 
most  adamantly  cling  to  this  objection  to  an  unwanted  conclusion,  jver 
hesitate  to  advance  the  sales  force  on  the  basie  of  their  "proven  record,  " 
That  these  two  attitudes  are  inconsistent  is  my  only  point. 

All  soothsayer  type  forecasting  of  the  future,  including  forecasts 
of  election  outcomes,  depends  for  its  acceptance  on  the  fact  that  the 
public  suffers  from  the  misapprehension  that  one  success  is  a  guaranty 
of  superhuman  powers,  Business  chooses  its  tycoons,  and  government 
its  "bright  young  men"  by  this  technique,  I  hasten  to  add  that  it 
guarantees  the  wrong  choice  no  better  than  it  does  ths  right.  It  may  be 
called  the  one  step  (two  etep,  k-etep)  beta  decision  rule, 

The  complexity  that  defeats  description  denies  certainty.  If  the 
administrator  must  act  today,  guided  only  or  principally  by  his  "intuitive 
grasp"  of  the  "total  situation"  and  not  on  the  basie  of  previously  enunciated 
rules  thought  to  embrace  the  situation  at  hand,  then  he  cannot  carry  over 
any  lesson  for  tomorrow  whether  he  is  successful  or  a  failure  today. 
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That  experience  confirms  error  as  often  as  truth  is  hardly  a  novel  observa- 
tion.  This  is  in  fact  the  beta  hypothesis.  One  does  indeed  learn  by  experi¬ 
ence,  but  at  present  only  by  an  extravagant  volume  of  repetition,  which 
then  forces  an  abstraction  into  the  consciousness  of  the  "man  of  action.  " 
Once  verbalized,  experience  can  confirm  or  refute  the  hypothesis.  Again, 
the  beta  hypothesis  asserts  that  experience  will  lead  to  the  explicit  formu¬ 
lation  of  wrong  inferences  quite  as  much  as  right.  But  just  as  bleeding 
as  a  medical  treatment  finally  gave  way  to  overwhelming  contrary  evid¬ 
ence,  co  must  any  belief  in  conflict  with  reality,  if  only  there  is  sufficient 
experience  since,  ultimately,  despite  the  influence  of  the  forces  described 
in  the  beta  hypothesis,  there  will  be  an  attrition  of  incorrect  deductions 
and  an  enrichment  of  correct  ones.  There  is  a  statistical  technique  to 
deal  with  such  cases --Sequential  Analysis--but  its  applicability  in 
administrative  contexts  must  so  far  have  occurred  to  only  one  individual, 

True,  the  practical  administrator  would  never  assemble  the  number 
of  observations  suggested  by  Sequential  Analysis  before  making  his 
decision--except  in  matters  of  trivial  moment.  He  is  thereby  protected 
by  (and  confirmed  in)  his  belief  that  his  poor  batting  average  belies  the 
very  quantitative  reasoning  he  ignores! 

Illusion  six  (Figure  10)  is  the  most  pervasive  of  all,  for  we  each 
see  the  world  only  from  our  own  point  of  view.  In  Adam  Smith's  day 
it  was  argued  that  the  interplay  of  universal  self  interest  would  produce 
a  general  maximum  of  social  well  being  through  the  action  of  an  "unseen 
hand.  "  It  has  since  become  clear  that  the  hand  is  either  unsteady  or 
unfriendly.  But  the  same  illusion  has  taken  refuge  in  the  cry  "let  the 
experts  decide.  "  Curiously,  it  is  in  the  military  field  where  the  dictum 
of  Clcmenceau:  "war  is  much  too  important  to  leave  to  the  generals" 
is  most  known  and  most  accepted,  Clemenceau  should  have  realised 
that  if  he  didn't  say  that  road  building  is  too  important  to  leave  to  the 
engineers  and  management  too  important  to  leave  to  the  managers  that 
no  successor  would  arise  capable  of  that  generalization.  The  experts 
should  be  left  to  their  own  devices --when  they  are  concerned  with  matters 
of  interest  only  to  themselves.  In  particular,  "research  managers"  are 
only  effective  when  they  adhere  to  Jeffersonian  principles  of  government 
(not  his  actions).  What  happens  when  diverse  activities  impinge- -each 
the  field  of  a  different  group  of  "experts -is  the  subject  of  illusion  seven. 
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*Used  by  special  permission  of  THE  SATURDAY  EVENING  POST. 
0  1964  by  The  Curtis  Publishing  Company. 
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The  "blight  of  the  expert"  takes  fwn  form*.  Ir.  tho  previous  para¬ 
graph  1  discussed  the  effect  of  letting  the  expert  "do  his  own  Job"  where 
it,  however,  impinges  on  others  and/or  on  their  capacity  to  do  theirs, 

The  second  form  is  to  extend  the  expertise  from  the  field  in  which  it  is 
earned  into  other  distinct  fields  where  it  is  not.  This  illusion  is  related 
to  but  quite  distinct  from  illusion  one.  In  "Peas  in  a  Pod,  "  the  assump¬ 
tion  is  that,  for  example,  every  engineer  is  interchangeable  with  every 
other  engineer.  This  second  form  of  the  "Tail  of  the  Dog"  illusion  is 
that  any  engineer  (physician,  military  officer,  educator,  or  bricklayer) 
and  only  an  engineer  (physician,  military  officer,  educator  or  brick- 
layer)  can  perform  any  other  function  regardless  of  the  skills  involved 
in  an  engineering  (medical,  military,  educational,  or  construction) 
organization.  The  example  that  caught  my  eye  here  is;  *  "1  think  all 
good  statisticians  agree- -and  I  define  a  good  statistician  to  be  one 
who  agrees--th&t  statistics  is  not  mathematics.  On  the  other  hand, 
it  happens  to  be  a  peculiar  subject  of  its  own,  which  mathematicians 
when  they  do  take  the  trouble,  can  teach  much  better  than  non-mathema- 
ticians,  " 

Illusion  seven  (Figure  11)  is  intended  to  conjure  up  in  the  viewer'* 
mind  not  just  the  genus  Rosa  but  the  whole  field  of  classification.  Classi¬ 
fication  is  associated  most  securely  in  the  popular  mind  with  the  fields 
of  zoology  and  botany- -but  is  somewhat  less  frequently  recognized  as 
a  universal  tool  of  science.  Languages,  rocks,  and  forms  of  tribal 
kinship  relations  are  classified,  The  most  famous  classification  In 
mathematics  is  that  of  Klein,  but  others  continue  to  Arise  in  every  quar¬ 
ter,  Second  only  to  biology  the  public  comes  in  contact  with  classifica¬ 
tion  at  the  public  library.  Librarians,  at  least  in  America,  will  street 
that  a  universal  classification  is  not  obtainable  -  -but  administrative 
practice  haa  not  yet  learned  this  lesson.  The  form  in  which  this 
illusion  enters  here  is  in  the  search  for  the  holy  grail  of  the  perfect 
organization. 


*John  G.  Kemeny  in  "New  Direction*  in  Mathematics,  "  proceedings  of 
conference  arranged  by  John  G,  Kemeny  and  Robin  Robinson,  Edited  by 
Robert  W.  Ritchie.  (Englewood  Cliffs ,  New  Jersey,  Prentice -Hall , 
1963).  I  strongly  suspect  that  Professor  Kemeny  made  the  above 
remarks  with  tongue  in  cheek  and  has  succeeded  in  pulling  my  leg, 

Why  else  would  he  commit  such  a  transparent  logical  non  sequitur? 
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That  a  satisfactory  organizational  classification  for  the  distribution 
of  responsibilities  has  not  yet  been  achieved  is  shown  by  the  fact  that  we 
are  forever  reorganizing,  What  1  wish  to  call  attention  to  here  is  that 
this  process  is  never  interrupted  to  check  on  the  existence  theorem,  nor 
to  seek  a  fruitful  setting  for  an  examination  of  the  problem.  Organisation 
is  the  replacement  of  the  disorganization  chart  by  the  organisation  chart, 
Bureaucracy  is  the  conversion  of  the  paths  in  the  disorganization  chart 
from  paths  of  persuasion  to  paths  of  coercion.  One  testimonial  to  the 
fact  and  to  the  consequences  was  the  establishment  by  the  Army  of  the 
"Program  Managers, 

In  the  Navy's  Polaris  Program,  (the  one  instance  success  which 
convinced  the  Army)  according  to  an  article  in  the  Civil  Service  Journal 
for  July-September  1964,  Admiral  Raborn  was  authorised  by  letter  to 
get  "whatever  people  and  whatever  cooperation  he  required  from  any  of 
the  Navy's  bureaus  and  offices.  "  Yet  "Admiral  Burke  admonished  him 
that  if  the  letter  ever  had  to  be  used  to  force  cooperation,  the  project 
would  fail,  "  Notice  that  Admiral  Raborn  went  out  of  channels,  he  used 
the  organisation  of  the  disorganization  chart,  he  used  these  lines  as 
lines  of  persuasion- -not  ae  lines  of  control,  This  is  where  organisa¬ 
tion  began! 

Procrustes'  bed  (Figure  12)  was  the  ancients'  way  of  putting  the 
principle  of  what  we  now  know  as  "the  organization  man,  "  In  America 
every  man  is  an  individual- -  "just  don't  step  out  of  line,  "  I  wish  to 
unite  this  principle  with  another  not  perhaps  widely  known  but  at  least 
not  due  to  me.  It  has  been  remarked  that  never  in  history  (of  course; 
seldom  in  history)  have  the  professionals  invented  a  radical  departure 
destined  to  lose  them  their  jobs, 

In  an  article,  "Revisionist  Theory  of  Leadership"  by  Professor 
Warren  G.  Bennis  in  the  Harvard  Business  Review  for  January-February 
1961,  Volume  39,  pages  26  ff,  the  author  on  page  148  observes;  "along 
these  lines,  Samuel  Goldwyn  was  reputed  to  have  said  to  his  staff  one 
day,  'I  want  you  all  to  tell  me  what's  wrong  with  our  operation- -even  if 
it  means  losing  your  Job)  '  "  I  thought  in  Hollywood  telling  the  boss  what 
he  was  doing  wrong  was  a  sure  way  to  lose  one  's  job.  I'm  sure  that 

*The  beta  virus  strikes  again!  It  is  something  of  an  achievement,  surely, 
that  bureaucracy  is  to  be  cured  by  increased  bureaucracy. 
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Professor  Bennis,  like  me,  thinks  this  story  is  aprocrophyl--but  it  At 
least  lists  one  impediment  to  people  with  an  assigned  mission  ever  making 
very  radical  changes  in  how  its  done, 

The  National  Road  starts  at  Braddock'e  Rock  near  the  Lincoln 
Memorial  and  runs  through  Frederick,  Maryland,  and  .then  on  west.  It 
has  been  said  that  the  waggoners  on  the  road  didn't  develop  the  canal 
system,  the  latter  didn't  develop  the  railroad.  No  railroader  developed 
the  automobile  or  the  airplane  - -indeed,  didn't  even  develop  the  diesel 
locomotive.  But  administrators  have  not  yet  drawn  the  obvious  conclu¬ 
sion.  If  a  department  is  charged  with  a  certain  responsiblity, .  then  that 
department  will  never  introduce  radical  changes.  If  anyone  does,  soma*; 
one  else  will  do  so,  But  so  soon  as  h«  doss,  he  will  have  the  oarganUa- 
tion  manual  thrown  at  him.  Anything  hs  may  have  dons  will  be  destroyed 
and  the  task  will  be  transferred  from  one  who  cares  and  knows  to  one 
who  fears  and  opposes.  At  least  that  was  my  experience,  and  of  several 
of  my  former  colleagues. 

This  consequence  is  particularly  unfortunate,  since  if  my  claim 
that  the  "research  manager  it  beet  who  manages  the  least"  is  correct, 
then  this  is  the  environment  which  will  yield  the  largest  number  of  out¬ 
standing  results.  But  the  consequence  of  illusion  seven  (A  Rose  le  a 
Rose  Is  a  Rose)  will  be  that  such  a  manager  will  get  a  low  rating  and  be 
accused  of  permitting  "excessive  duplication.  "  Even  here  I  can  claim 
no  priority.  Albert  Hirschman  and  Charles  Lindbloom  in  an  article 
"Economic  Development  Research  and  Development  Policy  Making; 

Some  Converging  Views,  »  in  BEHAVIORAL  SCIENCE,  Volume  7,  1962, 
pp.  211-222,  explicitly  recognise  the  benefits  of  a  little  "play"  in  the 
tight  constraints  placed  onresearch  and  development  efforts, 

Illusion  seven  (A  Rose  Is  a  Rose)  and  eight  (Procrustes'  Bed) 
together  are  at  the  bottom  of  much  of  the  ferment  over  "management" 
of  research.  It  seems  so  simple  to  divide  up  the  field  of  action  and 
portion  out  support,  responsibility,  and  facilities  to  each.  We  may 
even  take  a  leaf  from  the  military  field  commander,  and  recognising 
that  liaison  between  adjacent  commands  is  the  weak  point  in  a  battle 
line,  we  can  be  tolerant  of  a  certain  amount  of  interpenetration  at  the 
peripheries.  But  innovation  arises  from  the  darnedest  sources.  If 
t  arises  in  the  civilian  economy  or  at  least  from  non-governmental 
lurcet,  we  need  merely  create  an  Office  of  Scientific  Research  and 
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The  financial  pages  of  the  daily  papers  have  more  than  once  announced 
the  appointment  of  some  outstanding  administrator  as  president  of  a  com* 
pany- -sometimes  to  rescue  it  from  financial  difficulty,  The  aspect  I  wish 
to  draw  attention  to  here  is  that  in  a  number  of  such  instances  we  learn  ' 
that  the  former  wizard  has  now  been  replaced  by  a  new  one,  The  criteria 
by  which  administrative  competence  is  to  be  measured  seem  not  yet 
always  to  have  separated  sound  from  substance,  That  ie  is  more  impor¬ 
tant  to  look  right  than  to  be  right  is  the  foundation  of  thoae  two  esteemed 
aspects  of  business -advertising  and  selling.  It  is  also  the  foundation 
stone  of  politics  (at  least  of  the  office  variety),  No  author  of  fiction  or 
movie  director  has  the  slightest  uncertainty  as  to  the  great  separation.  .. 
between  appearance  and  reality--or  as  to  which  is  better  paid, 

My  final  illuaion  is  perhaps  the  most  subtle.  That  the  whole  is 
more  than  the  sum  of  the  parts  (Figure  14)  has  been  most  vehemently 
asserted  by  thoae  who  moat  abhor  quantitative  thinking,  But  the  claim 
has  been  more  often  used  as  a  bar  to  thinking  than  as  a  basis.  It  ie  the 
source  of  the  claim  "your  vote  does  count”- -though  I  think  not  for  the 
reason  usually  given.  The  true  reason  is  that,  if  the  public  would  ngrae,  . 
it  could  rule,  and  agreement  does  not  require  that  we  aseemble  an  over¬ 
whelming  force  ,  but  merely  that  we  inform  oureelvee,  Of  couree,  it 
doea  little  good  for  just  one  voter  or  a  few  voters  to  do  eo--the  answer 
liea  In  the  whole  (nearly)  doing  so.  Hence  it  doesn't  occur.  Hence  the 
principle. 

This  fallacy  has  been  at  the  root  of  the  argument  over  international 
trade  since  the  Industrial  Revolution.  Adam  Smith  wrote  hla  book  to 
set  the  argument  right,  and  while  he  did  convince  the  economist,  hs 
didn't  convince  anyone  who  counted.  Our  individual  fortunes  ars  con¬ 
trolled  primarily  by  our  fate  as  producers,  but  our  collective  fortunee 
are  equally  involved  as  consumers,  Since  the  ordinary  parson  considers 
his  individual  (his  producer)  function  first  and  foremost  he  penalises 
himself  and  his  fellow  citizens  by  supporting  protectionism,  The  most 
widely  recognized  recent  large  scale  instance  of  this  illusion  has  been 
exposed  by  John  Maynard  Keynes  in  connection  with  governmental  fiscal 
and  monetary  policy.  Fortunately,  some  of  his  resulting  conclusions 
can  be  supported  on  valid  groundc, 

Public  support  of  law  enforcement  "crackdowns”  or  of  any  other 
mitive  or  control  measure,  e,g.  ,  traffic  crackdowns,  compulsory 


Design  of  Experiments 


453 


vehicle  inspection,  depends  in  large  part  on  an  assumption  that  it  is 
always  "the  oth«r  guy"  who  will  be  affected.  The  risk  of  personally 
being  victimized  is  small.  But  it  is  not  zero.  And  tho  consequences 
(where  the  action  is  misguided)  injures  society  as  a  whole--over  and 
above  the  fact  that,  of  course,  while  wrong  actions  are  in  progress, 
right  actions  must  wait. 

An  exemplification  closer  to  our  topic  and  of  vital  importance  lies 
in  the  field  of  experience.  No  one 's  own  experience  is  "whole"  till 
completed,  whether  of  an  individual,  a  firm,  or  a  nation.  Hence  one 
cannot  profit  by  experience  as  a  whole  while  he  can  yet  experience  it, 

If  the  whole  differs  from  the  sum  of  the  parts,  it  will  have  no  effect. 

But  the  very  meaning  of  a  "rare"  event  is  that  a  brief  experience, 
possibly  even  that  of  a  life  time  (whether  of  a  person  or  of  a  nation) 
is  not  long  enough.  It  has  been  claimed  that  while  a  fool  profits  by  no 
one's  experience  and  an  ordinary  man  only  by  his  own,  a  wise  man 
profits  by  everyone's.  Yet  even  a  wise  man  cannot  profit  by  experience 
he  doesn't  know  about,  And  there  ere  many  fields  in  which  experience 
is  so  rare  that  it  is  next  to  impossible  to  assemble  it  (1.  e.  complete  it). 
The  Constitution  says  the  right  of  the  people  to  keep  and  bear  arms  shall 
not  be  abridged.  This  was  based  on  experience.  But  no  living  man  can 
have  had  their  experiences ,  and  how  to  apply  them  to  modern  conditions 
it  not  obvious.  Every  teen-ager  begins  with  much  confidence  and  little 
experience  and  ends  with  more  experience  than  confidence,  That  gang¬ 
sters,  racing  drivers,  entrepreneurs,  dictators,  explorers,  and 
inventors  base  their  future  actions  on  their  own  past  but  necessarily 
incomplete  experience  is  insufficiently  recognized,  Had  all  past  experi¬ 
ence  been  considered,  many  of  these  occupations  would  become  deserted. 

The  constant  advice  to  the  young  and  gullible  to  "take  a  chance"  in 
choosing  a  career  and  so  secure  an  opportunity  to  win  a  handsome 
financial  reward  is  not  seen  to  be  at  one  with  advice  to  gamble  on  the 
stock  market- -or  on  a  slot  machine,  And,  of  course,  the  proapec;  of 
a  material  reward  (or  fear  of  material  failure)  ie  assumed  to  motivate 
all  choices  in  our  society. 

That  statisticians  are  professionals  at  extracting  the  lessons  of 
others'  experience  is  recognized  in  a  few  circumscribed  fields  like 
,'esearch,  development,  and  testing,  or  quality  control,  or  acceptance 
■•mpling,  but  not,  it  seems,  in  opinion  polling--or  in  safeguarding  the 
i  of  a  president. 
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My  final  figure  enumerates  these  ten  quantitative  illusions  ot 
administrative  practice  It  is  my  belief  that  if  administrators  generally, 
almost  universally,  acquired  an  instinctive  capacity  to  recognize 
instances  of  any  of  them  on  sight,  despite  their  Infinite  capacity  for 
disguise  or  fragmentary  manifestation,  then  rational  administration 
just  might  some  day  be  possible. 


TEQUILAP 

1.  Peas  in  a  Pod 

2.  Profit  and  Loss 

3.  So  Much  the  Better 

4.  That's  Enuf,  Cusp  It 

5.  A  One  Dimensional  View 

6.  Tail  of  the  Dog 

7.  A  Rose  Is  a  Rose  is  a  Rose 

8.  Procrustes 1  Bed 

9.  Sound  and  Substance 

10.  Whole  and  Part 

FIGURE  15 


COMBAT  VEHICLE  FLEET  MANAGEMENT 


C.  J.  Christianson  and  G.  E.  Cooper 
Research  Analysis  Corporation,  McLean,  Virginia 


Over  the  last  six  years  the  Operations  Research  Office  and  the  Re¬ 
search  Analysis  Corporation  have  studied  a  variety  of  US  Army  vehicles 
from  a  maintenance -oriented  point  of  view.  All  the  studies  have  held  the 
fundamental  premise  that  to  a  greater  extent  many  Army  materiel  deci¬ 
sions  and  programs  should  be  based  on  certain  mechanical  properties  of 
materiel  in  the  real  troop-machine  environment.  In  the  past  the  rarity 
of  pertinent,  real  data  too  often  resulted  in  the  mechanical  properties' 
being  ignored  or  unrealistically  appraised.  It  was  apparent  that  many 
performance  objectives  were  far  from  adequate  indicators  of  actual 
achievement.  ORO  then  and  RAC  now  have  sought  to  introduce  realistic 
measures  of  mechanical  effects  into  managerial  decision  processes,  Of 
necessity  this  management  research  mission  had  to  be  confined  to  lim¬ 
ited  numbers  and  types  of  equipment.  In  order  to  determine  meaningful 
equipment  policies,  it  is  necessary  to  consider  troop-performance  data 
in.  combination  with  a  variety  of  monetary  costs  and  with  relative  obsoles¬ 
cence,  Neither  dollars,  nor  obsolescence,  nor  mechanical  performance 
alone  can  be  expected  to  give  infallible  guidance  to  the  necessary  consid¬ 
eration  of  equipment  management.  Occasionally  technological  break¬ 
throughs  do  occur,  and  sometimes  a  particular  equipment  model  does 
develop  a  rash  of  breakdowns.  However,  in  the  long  run  the  Army  has  to 
program  much  of  its  inventory  between  the  relatively  gradual  changes  in 
the  designs  for  new  production  and  the  mechanical  aging  of  previously 
manufactured  equipment. 

The  most  current  study  of  Army  vehicles  has  been  selected  as  a 
general  example  of  the  ways  in  which  RAC  has  contributed  both  greater 
qualitative  understanding  and  improved  numerical  assessment  techniques 
to  the  solution  of  Important  materiel  management  problems.  Certain  of 
the  results  have  been  disguised  for  relatively  open  presentation;  however, 
wherever  possible,  numerical  examples  have  been  kept  along  the  scales 
of  their  true  values.  Indeed,  there  is  often  a  need  to  force  experts  to 
recognize  the  order  of  magnitude  of  the  effects  with  which  they  profess 
intimate  qualitative  acquaintance. 

■\l — The  Research  Analysis  Corporation  recently  completed  a  comprehen- 
|ve,  multi-stage  analysis  of  three  different  types  of  tracked  vehicles 
'.•rated  and  maintained  in  US  Army  combat  units  in  Europe.  A  main 


456 


Design  of  Experiment* 


objective  of  such  itudy  was  to  determine  in-uee  live*  of  tank*,  armored 
personnel  carriers,  and  recovery  vehicle*.  In  addition  to  fulfilling  it* 
principal  objective,  the  study  provided  a  variety  of  by-product* 

essential  to  successful  management  of  combat  vehicle  fleets.  Measure¬ 
ment  of  fleet  wearout,  establishment  of  repair  capacity  requirement*,  pro¬ 
jection  of  budget  needs,  assessment  of  combat  readiness,  and  determina¬ 
tion  of  fleet  replacement  factors  (procurement  requirements)  are  all 
management  responsibilities  that  cannot  be  successfully  fulfilled  without 
basic  information  and  analytical  results  of  the  type  provided  by  RAC. 

PROBLEM 

COMBAT  VEHICLE  FLEET  MANAGEMENT 

Utilization 

Component  Replacement  Forecasting 

Readiness 

Study  conducted  for  the  U.  S,  Army 
by  the  Research  Analysis  Corporation 

Figure  1 


Inasmuch  as  the  same  analytical  procedures  were  applied  by  the  study 
to  all  three  vehicle  types,  little  generality  is  sacrificed  if  most  further 
discussion  is  limited  to  just  one  vehicle  - -the  tank,  The  determination  of 
in-use  lives  is  treated  here  as  only  a  part  of  the  general  problem  of  tank 
fleet  measurement,  Within  a  manageable  framework  equipment  life  is 
interrelated  with  factors  of  utilization,  component  replacement  forecasting 
and  costing,  and  materiel  readiness,  to  name  only  a  few.  The  factors 
cannot  be  divided  into  one  list  of  independent  and  another  of  dependent 
variables,  They  can  be  functionally  related,  but  any  adopted  conventions 
of  dependence  and  independence  are  likely  to  be  unrealistic  statements  of 
causes  and  effects, 

In  all  RAC  equipment  studies  the  starting  point  of  meaningful  concepts 
has  always  been  a  body  of  carefully  collected  empirical  data,  The  hypoth- 
eses  preceding  observation  have  almost  always  had  to  be  so  modified  unde'" 
the  light  of  experience  that  it  now  proves  preferable  to  avoid  prejudicing^ 
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early  theories.  The  logical  elegance  of  classical  scientific  method  has  to 
be  modified  in  favor  of  diffusely  related  series  of  observations  resulting 
in  near -saturation  with  both  pertinent  and  extraneous  data.  Data  process¬ 
ing  and  analysis  then  have  to  be  not  so  much  testing  of  theory  as  they  must 
be  sorting  of  the  relevant  from  the  irrelevant.  Of  necessity  the  rules  of 
relevancy  must  be  continually  modified  as  more  and  more  data  are  digested. 
And  discouragingly  it  often  becomes  essential  to  completely  reprocess 
earlier  work. 

Fig.  2 — The  body  of  empirical  data  used  for  the  most  current  study  accrued 
over  the  complete  history  sample  is  now 'shown.  Maintenance  events  that, 
occurred  against  all  these  vehicles  were  recorded  by  vehicle  number,  age, 
and  mileage.  The  column  on  the  right  gives  ample  evidence  of  the  fact 
that  large  numbers  of  combat  vehicles  are  used  extensively  in  peacetime. 
Peacetime  is  unquestionably  a  time  of  appreciable  materiel  consumption. 
That  the  severity  of  peacetime  demands  is  still  not  universally  recognized 
in  all  related  civilian  and  military  agencies  continues  to  provide  one  of 
the  major  obstacles  to  completely  successful  combat  vehicle  management. 

SAMPLE 


No.  of 
vehicles 


Months 

observed 


Average 


Miles 
observed 


L  sage 

miles  per  mo. 


Tank 

640 

21 

2739 

130 

Armored  Personnel 

Carrier 

708 

18 

2655 

148 

Tank  Recovery  Vehicle 

83 

19 

1485 

78 

Figure  2 


Fig.  3; — One  of  the  fundamental  operations  performed  on  the  body  of  empirical 
data  is  the  construction  of  mileage -dependent  parts  costs.  The  average 
direct  costs  for  tank  repair  parts  are  shown  for  each  successive  500-mile 
increment  to  4500  miles.  Track  and  engine  replacements  account  for 
most  of  the  dollar  consumption.  One  of  the  most  important  features  of 
arts  consumption  is  the  strong  increase  with  mileage  beyond  1500  miles. 
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Fig.  3- Average  Coat  Per  Tank  by  500-Mile  Increments 
Total  Miles  Obaerved  1,75  Million 
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Through  1500  miles  parts  expense  averages  very  close  to  $1  per  mile. 
Beyond.1500  miles  both  track  and  engines  enter  pronounced  replacement 
phases.  Modal  track  replacement  occurs  at  about  2200  miles.  The  dollar 
effect  of  this  mode;is  to  drive  parts  costs  above  $8  per  mile  in  the  2000.-. 
to  2500-mile  interval.  Beyond  2500  miles  there  is  a  definite  decline  in 
expense,  but  a, second  generation  of  track  replacement  beyond  3500  miles 
forces  the  costs  up  again.  From  a  separate  examination  of  the  same 
basic  data  it  was  estimated  that,  if  all  subsequent  installations  of  parts 
and  assemblies  provided  lives  similar  to  those  of  the  originally  installed 
parts,  an  equilibrium  cost  would  just  exceed  $6  per  mile  for  a  list  of  the 
principal  mobility-affecting  parts.  Inclusion  of  weapon  and  fire  control 
costs  would  add  to  the  $6  per  mile  figure.  The  changes  in  parts  con¬ 
sumption  with  mileage  obviously  have  tremendous  impact  of  the  provision 
and  budgeting  of  parts  and  maintenance  support.  If  tanks  are  operated 
at  1500  miles  per  year,  support  during  the  first  year  of  tank  life  be  only 
one-sixth  that  required  during  an  equilibrium  year.  Too  much  support 
planning  is  still  dependent  on  an  assumption  that  each  new  year  is  going 
to  be  like  the  last  one.  The  importance  of  making  predictions  based  on 
analyses  of  trends  is  revealed  by  data  of  the  cost  type. 

Fig.  4 — For  the  same  tank  sample,  the  purely  historical  cost  average  to 

nearly  two  years  was  $3.  75  per  mile.  The  truth  of  such  history  does  not 
make  the  past  a  guaranteed  base  for  prediction  of  identical  futures.  Vehicle 
support  must  be  programmed  in  calendar  time.  In  order  to  provide  accu¬ 
rate  support  it  is  necessary  to  know  both  the  basic  consumption  rates  per 
mile  and  the  mileages  to  be  covered  during  given  calendar  intervals. 

Again  in  the  case  of  the  tank  sample,  adjustment  of  the  average  mileage 
cost  to  the  average  use  rate  yields  monthly  costs  of  $489  per  tank  per 
month.  At  the  same  rate  of  travel,  but  using  tanks  with  much  higher 
mileages,  corresponding  expenses  would  exceed  $780  per  tank  per  month. 


Figure  4 

PARTS  COSTS  FOR  TANKS 
(Usage:  130  miles  per  month) 


Cost  per 

Cost  per 

mile 

month 

Engines’1' 

$1.19 

$155 

T  ransmissions* 

.19 

25 

Other  Parts 

.  32 

42 

Track- -amortized  at  $2.  05  per  mile 

2.  05 

267 

Total 

30  percent  of  list  price. 

$3.  75 

$489 
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Fig.  5 — The  occurrences  of  replacement  events  are  more  fundamental  than 
their  associated  costs.  One  of  the  advantages  of  the  format  employed  for 
collection  of  the  basic  history  data  is  that  mortality  statistics  for  individual 
parts  types  can  be  determined  by  straightforward  actuarial  calculation.  For 
example,  the  comulative  engine  replacement  experience  is  shown  for  the 
total  tank  sample.  The  accumulation  of  replacements  increases  non-linearly 
to  just  beyond  2000  miles,  but  then  the  engine  activity  becomes  almost 
uniform. 

Fig.  6 — The  replacement  of  original  engines  was  interpreted  as  having  been 
governed  by  an  underlying  distribution  like  that  labelled  first.  That  distri¬ 
bution  is  represented  by  two  distinct  phases.  The  first,  extending  to  just 
beyond  1000  miles,  averages  only  about  one -half  percent  replacement  per 
100  miles.  This  first  phase  corresponds  closely  to  the  typical  early 
"debugging"  period  so  common  to  many  kinds  of  equipment.  However, 
unlike  the  frequently  discussed  "bath-tub"  effect,  tank  engines  do  not  then 
experience  a  phase  of  reduced  replacement  activity.  Rather,  the  tank 
engines  immediately  being  a  second  phase  with  sharp  increase  in  the 
replacement  rate.  The  second  phase  corresponds  to  what  one  would  expect 
with  entry  into  a  wear-like  mode  of  engine  mortality.  The  presented 
distribution  of  mileages  to  replacement  has  a  mean  of  3700  miles. 

If  it  is  assumed  that  all  succeeding  engine  installations  will  result 
in  lives  like  those  of  the  original  engines,  second  and  third  replacements 
will  occur  as  shown  by  the  distributions  labelled  2d  and  3d  in  the  figure. 

Such  higher  order  replacements  have  been  determined  by  taking  repeated 
convolutions  of  the  first  distribution.  The  sum  of  replacements  of  all 
orders  is  often  described  as  the  renewal  density.  It  represents  the 
instantaneous  replacement  rate  disregarding  the  order  of  a  replacement. 

A  pure  coincidence  of  the  chosen  first  replacement  distribution  is  that  its 
corresponding  renewal  density  approaches  equilibrium  quickly  but 
smoothly.  In  fact,  by  3000  miles  the  renewal  density  is  almost  equal 
to  its  equilibrium  value.  The  absence  of  strong  oscillations  in  replace¬ 
ment  activity  is  usually  of  great  advantage  in  the  prediction  of  maintenance 
support  requirements.  Somewhat  later  comments  will  be  made  by  way  of 
explanation  of  real  effects  that  tend  to  drive  engine  replacement  activity 
above  the  so-called  equilibrium  presented  here. 

Fig.  7 — Conversion  of  a  renewal  density  of  corresponding  budget  costs  is  a 
relatively  easy  matter.  Multiplication  of  the  renewal  density  by  the  unit 
cost  of  a  replacement  yields  the  desired  measure  of  expense.  As  an 
example  of  the  type  of  results  derived  by  such  a  processing  operation, 
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a  cost  rate  dependent  on  mileage  is  shown  here  for  engine  and  transmission 
replacement  activity.  Such  information  represents  interpretation  and 
extrapolation  of  the  originally  collected  empirical  data.  Transmissions 
have  lower  unit  prices  and  longer  lives  than  do  tank  engines.  This  double 
advantage  to  the  transmissions  makes  their  costs  per  mile  considerably 
lower  than  that  of  the  engines.  Note  that  the  total  expense  for  these  two 
assemblies  accounts  for  nearly  one-half  the  previously  mentioned  equilib¬ 
rium  cost  rate  of  roughly  $6  per  mile. 


Cost/500  Miles  Cost/Mile 

$1500  —  -  $3 


Mile  s 

Figure  7 --Tank  Major  Assembly  Replacement 
Cost  "Like  New,  "  30%  of  List  at  130 
Miles /Month 
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Reality  usually  has  a  way  of  muddying  the  clear  waters  of  analysis. 
The  preceding  engine. and  transmission  predictions  are. all  founded  on  an 
assumption  of  performance  from  replacement  assemblies,  like  that  of  the 
originals.  Unfortunately  very  little  data  have  accrued  about  the, perform¬ 
ance  of  replacement  engines  in. the  model  tank  most  recently  studied. 

Fig.  8— However,  the  experiences  of  a  preceding  model  tank  and  its  engines 
are  not  ignorable.  There  survivor  curves  are  . shown  in  the  projected 
figure.  All  these  were  determined  for  the  other  tank.  That  tank's 
original  engines  lasted  as  shown  by  the  line  labelled  "new.  "  Engines 
that  were  overhauled  by  a  depot  maintenance  facility  survived  only  as 
long  as  shown  by  the  curve  labelled  "overhauled.  "  Other  engines  were 
repaired  by  a  mobile  team  (fourth  echelon  in  the  US  Army).  The  lives 
of  engines  repaired  by  that  team  were  even  shorter  and  are  represented 
by  the  line  labelled  "re-ring  and  de -glaze.  "  After  several  years  virtually 
all  the  engines  that  are  installed  in  used  tanks  are  repaired  ones.  The 
possibility  must  be  recognized  that  equilibrium  engine  replacement 
activity  may  run  twice  as  high  with  repaired  engines  as  that  determined 
from  the  originally  performing  assemblies. 

Fig.  9 — For  the  current  model  tanks,  second  engine  replacements  have  been 
running  somewhat  higher  than  expected.  Only  about  10  percent  of  the 
second  engines  are  known  to  have  been  repaired  ones.  The  rest  are 
presumed  to  have  been  issued  from  storage  in  unused  condition.  If 
the  second  engines  were  to  do  as  well  as  the  originals  did  only  4  percent 
of  the  tanks  would  experience  a  second  replacement  by  3000  miles.  The 
predicted  "like -new"  experience  is  represented  by  the  lower  solid  line 
in  the  accompanying  figure.  Empirical  experience  is  shown  by  the  lower 
dotted  line  and  runs  just  over  8  percent  replacement  to  3000  miles.  The 
real  replacement  mechanism  seems  to  be  one  that  is  only  partly  renewal¬ 
like.  Tank  age  regardless  of  assembly  age  appears  to  provide  an  impor¬ 
tant  component  of  the  probability  of  replacement.  By  now  listeners  are 
probably  wondering  how  the  pure  renewal  model  can  be  of  any  use  of 
management  if  so  many  of  reality's  disturbing  influences  tend  to  raise 
activity  much  higher.  The  renewal  model  provides  a  valuable  reference 
point.  First  of  all,  the  renewal  estimates  have  already  revealed  that 
many  support  programs  are  set  even  lower  than  these  predictions.  The 
pure  renewals  usually  specify  the  best  that  can  be  expected.  If  support 
is  too  low  and  is  not  geared  to  satisfy  the  demands  of  the  best  possible, 
it  should  be  raised  immediately  to  at  least  the  lewl  consistent  with  a 
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renewal  prediction,  And  eemnd,  the  renewal  e»simates  provide  the  basis 
01  comparison  of  performance  of  repaired,  stored,  or  modified  assemblies 
with  original  quality,  Too  often  statements  are  made  that  some  component 
is  good  or  bad  without  expressing  goodness  or  badness  relative  to  a  real 
mechanical  standard.  The  usual  comparisons  with  paper  standards  have 
caused  more  confusion  than  they  have  eliminated,  Even  original  engine* 
do  not  compare  favorably  with  their  paper  standards  by  which  they  were 
designed,  built,  and  procured  for  Army  u#e.  Although  performance  of 
materiel  in  the  hands  of  troops  cannot  be  the  ideal,  absolute  standard  of 
reference,  it  remains  the  best,  most  meaningful  basis  of  comparison  at 
the  present  time. 


-  Theoretical 

Empirical 


Figure  9--Comparison  of  History  with  Expected 
Like-New  Renewals  of  Engines 
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Fig.  10 — The  replacement  distributions  oi  most  parts  ana  assemblies  are  widely 
distributed  about  their  means.  A  particularly  important  example  of  a  part 
that  does  not  have  this  property  is  tank  track.  The  lighter  line  in  the 
projected  figure  shows  the  cumulative  replacement  experience  for  sets  of 
tank  track,  The  S- shaped  curve  represents  the  most  compact  distribution 
of  replacements  discovered  in  the  whole  series  of  studies  of  ground 
vehicles.  A  complete  set  of  track  is  expensive,  At  the  same  time  it  has 
the  shortest  life  among  all  the  high-unit-priced  parts  of  a  tank,  Conversion 
of  life  and  unit  price  to  costs  per  mile  yields  an  estimate  of  $2,  05  per  mile 
for  tank  track.  Although  engines  and  transmissions  coat  more  than  track, 
their  lives  are  sufficiently  long  to  make  their  mileage  costs  lower  than 
that  of  track,  Track  expense  accounts  for  roughly  one-third  the  predicted 
equilibrium  parts  costs  for  the  mobility-affecting  systems  of  tanks.  A 
cumulative  curve  for  second  replacements  is  also  shown,  RAC  pointed 
out  that  the  early  climb  of  the  second  curve  was  in  part  attributable  to  the 
use  of  much  shorter-lived  rebuilt  track. 

Fig.  11 — Much  effort  has  been  directed  toward  interpretation  of  the  empirical 
history  relative  to  notions  of  materiel  readiness.  No  one  definition  has 
proved  satisfactory  as  a  full  description  of  readiness,  but  several  less 
general  notions  have  proved  particularly  useful.  One  concept  that  has 
been  extremely  helpful  is  that  of  "equipment  availability  potential,  11  An 
item  of  equipment  is  considered  to  be  assignable  to  one  of  several  States 
of  serviceability.  For  example,  consider  the  four  states  shown.  An 
item  can  be  serviceable  or  it  can  be  in  one  of  three  (or  more)  states  of 
unserviceability.  That  item  can  undergo  transitions  from  one  state  to 
another  with  probabilities  associable  with  each  particular  type  of 
transition.  The  "k's"  can  be  associated  with  breakdown,  and  the  "X's" 
can  be  related  to  correction  or  repair  times,  The  transition  probabilities 
may  depend  on  ages,  mileages,  generation  numbers,  or  other  factors. 

Too  often  it  is  not  possible  to  detail  all  the  inter-relations,  In  fact  much 
of  the  time  the  greatest  utility  arises  from  considering  a  two-state  model; 
that  is,  one  state  of  serviceability  and  only  one  state  of  uniervicaabillty. 

The  instantaneous  "availability  potential"  is  defined  as  the  probability  of 
being  in  the  serviceable  state  if  transition  probabilities  were  to  permanently 
retain  their  current  value*,  When  the  transition  probabilitie a  change 
sufficiently  slowly,  the  availability  potential  gives  a  suitable  measure  of 
actual  current  serviceability  or  availability.  Constant  levels  of  availa¬ 
bility  potential  may  be  represented  by  hyperbolas  on  a  "k-X"  plane, 
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Figure  11 


Fig.  12— Analyeia  of  the  mobility-affecting  parte  replacement  data  from  the 
same  tank  history  sample  led  to  the  mileage  dependent  availability  path 
shown.  During  the  entire  history  period  the  average  response  time 
stayed  very  cloeo  to  5.  7  days  per  replacement  Job.  However,  because 
the  rate  of  replacements  per  mile  was  changing,  the  availability  at  125 
miles  per  tank  per  month  had  to  change  as  shown.  During  their  early 
lives  the  studied  tanks  were  operated  with  close  to  0,  98  availability 
potential.  As  they  accumulated  additional  mileage,  the  tanks  lost 
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-viilability .  During  the  period  of  rr.?!t  tr*rk  replacement* .  the  availa¬ 
bility  dropped  to  about  0.  95.  Then  followed  a  period  of  tome  improvement; 
the  availability  climbed  to  0.  95,  However,  beyond  3250  mile*  availability 
again  dropped  and  reached  about  0,  91  in  the  interval  4000  to  4500  mile*, 

Fleet  availability  need  never  be  coniidered  an  unmanageable  aspect 
of  operation  and  maintenance.  Rather  it  ia  only  a  result  of  several 
directly  manageable  factors.  Product  quality,  support  response  time, 
and  rate  of  equipment  use  all  affect  equipment  readiness. 

Fig.  13 — A  great  deal  of  attention  is  usually  given  to  the  supposed  or  predicted 
differences  in  product  quality  among  different  models  of  equipment,  The 
normal  approach  of  salesmanship  is  to  promise  that  some  new  model  will 
provide  mechanical  advantage!  far  beyond  the  cabilii.it*  of  its  predecessor. 
Too  often  the  demonstrated  comparisons  examine  an  unused,  new  model 
and  an  over-used,  old  model.  The  RAC  studies  have  discovered  that  many 
models  of  different  generations  appear  so  different  only  if  they  are 
examined  at  different  stages  in  their  lives.  At  the  same  ages  different 
models  of  tanks  possess  greater  similarity  with  respect  to  parts  replace¬ 
ment  rates  than  do  tanks  of  a  single  model  at  different  ages.  Judicious 
utilization  of  a  particular  tank  model  can  increase  overall  mechanical 
capability  more  than  can  a  transition  of  models  amid  a  less  carefully 
designed  program  of  vehicle  use. 

As  an  example  far  less  pronounced  than  reality,  consider  this  figure. 
Suppose  that  at  a  uniform  rate  of  use  some  tank  model  has  the  renewal 
density  shown  with  respect  to  time.  That  density  Increases  smoothly.  A 
change  of  utilisation  can  have  a  three-fold  effect  with  respect  to  time.  An 
increase  in  use,  in  effect,  squeezes  time  by  having  the  higher  accumulated 
mileages  occur  that  much  sooner  in  time.  Thus,  in  a  given  month  the 
replacements  per  mile  are  higher.  Then  because  more  miles  are  traveled 
during  that  month,  the  replacements  during  a  giver,  month  are  given  a 
second  boost.  The  third  effect  may  be  to  somewhat  alter  the  mileage 
lives  depending  on  the  rate  of  use.  It  is  not  illogical  to  consider  the 
possibility  of  increasing  or  decreasing  mileage  lives  depending  on  the 
type  of  component  involved.  Actually  it  is  usually  sufficient  to  suppose 
that  the  occurrence  of  events  per  mile  depends  on  the  accumulated  mileage 
but  not  on  the  rate  of  use,  This  assumption  is  nearly  correct  within  the 
mileage  ranges  normally  encountered  and  corresponds  to  having u)  equal 
to  1.  0. 


47  5 


ation  Effect 

o 

'  T 

in 


476 


Design  of  Experiments 


Fig,  14 — Now  consider  an  example  from  the  real  life  of  the  studied  tank*.  In 
the  accompanying  illustration  the  calculated  renewal  density  for  engine 
replacements  is  shown  along  a  time  base  of  tank  use  at  130  miles  per  tank 
per  month,  At  that  rate  of  use  engine  replacements  at  20  months  amount 
to  about  4.  6  per  100  tanks  per  month,  Now  consider  what  would  happen 
if  use  were  increased  50  percent  to  195  miles  per  tank  per  month.  At  20 
months  the  faster  tanks  would  have  the  same  total  mileage  as  do  the  slower 
tanks  at  30  months.  At  that  mileage  the  engine  replacement  rate  per  mile 
is  higher  than  at  the  lower  mileage,  At  the  same  time  the  tanks  are  going 
1,  5  times  as  many  miles  per  month,  The  net  result  is  that  at  20  months 
tanks  going  only  1,  5  times  as  fast  may  be  expected  to  experience  over  1.  8 
times  as  many  replacements. 

Next  consider  the  prospect  of  having  operated  those  same  tanks  at 
only  one-half  of  130  or  65  miles  per  tank  per  month,  At  20  months  the 
slower  tanks  have  accumulated  only  asrmany  miles  as  had  the  130-mile- 
per-month  tanks  at  10  months,  Thus  at  20  months  the  slower  teinks, 
experience  engine  replacement  at  a  much  lower  rate  par  mile,  and 
because  the  slower  tanks  cover  only  hiff  as  many 'miles  per  month,  their 
engine  activity  la  even  lower,  In  fact,  at  20  months  the  reduction  of  use 
by  one-half  results  in  only  0,17  times  as  many  engine  replacements, 

The  magnitudes  of  change  with  respect  to  time  that  can  be  effected 
by  utilisation  control  are  obviously  greater  than  many  of  tha  differences 
asserted  to  exist  among  different  models.  It  should  also  be  obvious  that 
the  utilisation  impact  extends  to  materiel  readiness,  assembly  repair, 
assembly  floats,  maintenance  allocations,  and  so  on  throughout  much  of 
fleet  management, 

Fig,  15— A  nomogram  was  constructed  as  an  illustration  of  the  utilisation 

effect  on  major  assembly  maintenance  activity.  The  nomogram  expresses 
the  interdependence  among  mlleagt  raplacamsnt  rates,  equipment  utiliza¬ 
tion,  time  replacement  rates,  durations  of  repair  pipelines,  and  assembly 
float  requirements. 

For  example,  consider  an  assembly  that  is  being  replaced  at  a  rate 
of  one  per  3700  miles  (reference  the  point  along  the  lower  left  scale), 

A  vertical  trace  to  roughly  130  miles  per  month  reveals  that  1000  tanks 
experience  about  38  replacements  of  that  assembly  per  month.  If  three 
months  are  required  between  the  removal  of  the  assembly  to  the  time 
that  it  is  repaired  and  available  for  re-use,  It  is  necessary  to  keep  Just 


Fig.  15- Relation  Among  Assembly  Life,  Rate  of  Use,  Duration  of  Unserviceableness  and 
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over  100  of  those  assemblies  in  the  assembly  float  or  pipeline.  Efficient 
exploitation  of  combat  vehicle  resources  demands  that  close,  continuous 
attention  be  given  to  all  the  quantities  described  in  the  nomogram.  At  a 
given  time  the  fleet  generates  demands  for  assembly  replacements  in  a 
way  that  depends  on  both  the  assembly  quality  and  the  fleet  use.  The 
duration  of  the  pipeline  depends  on  heavy  maintenance  programs,  stocks 
of  parts,  and  the  geographic  location  of  facilities.  A  trans -Atlantic  pipe¬ 
line  by  surface  transport  can  easily  run  to  many  months  and  result  in 
gigantic  increases  in  the  required  float  size. 

Fig.  16 — The  aggregate  of  mobility-affecting  parts  replacements  provides  a 
basic  indicator  of  what  to  expect  in  the  way  of  vehicle  performance.  Such 
data  were  already  used  to  determine  the  mileage  dependent  availability 
potentials  at  a  given  rate  of  use.  Perhaps  a  more  fundamental  way  of 
viewing  the  replacement  activity  is  to  consider  the  average  miles  per 
parts  replacement  action  over  a  range  of  mileages.  Such  information  is 
provided  in  the  projected  figure.  To  1500  miles  the  studied  tanks  per¬ 
formed  with  about  one  replacement  action  per  1000  miles.  Beyond  1500 
miles  the  performance  dropped  rapidly  with  the  incidence  of  a  great  deal 
of  track  replacement  activity.  Improvement  occurred  in  the  3000-  to 
3500-mile  range,  but  then  a  decline  again  appeared.  From  the  detailed 
basic  data  it  was  estimated  that  the  trend  would  eventually  lead  to  an 
equilibrium  activity  close  to  165  miles  per  replacement  job  for  the 
mobility -affection  parts.  This  level  is  based  on  the  assumption  that 
all  installed  parts  and  components  last  as  well  as  did  the  originals. 

Fig.  17 — The  figure  now  presented  provides  an  example  of  the  effect  of  using 
repaired  assemblies  as  replacements  in  older  tanks.  Tank  A  represents 
the  model  studied  most  currently.  The  tanks  down  for  engine  or  transmission 
replacement  are  shown  for  Models  A  and  B  when  all  replacement  assem¬ 
blies  perform  as  well  as  did  their  originals.  Model  B  was  actually  studied 
several  years  ago,  and  considerable  data  were  collected  for  it  during 
periods  when  it  did  receive  repaired  assemblies.  Model  B  was  operated 
at  a  much  lower  rate  of  travel,  but  translation  of  its  major  assembly 
experiences  to  the  same  tank  use  rate  as  that  of  Model  A  led  to  the  much 
steeper  line.  In  other  words,  had  the  older  model  tanks  been  operated  at 
130  miles  per  tank  per  month,  at  40  months  they  would  have  most  likely 
experienced  about  a  5-percent  deadline  rate  for  engine  or  transmission 
replacements  assuming  enough  assemblies  would  have  been  available  from 
the  repair  facilities.  Model  A  was  not  observed  much  beyond  20  months 
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and  hence  had  not  had  an  opportunity  to  experience  u*e  *ith  mo«tly  repaired 


w**  ■*  • 


larity  between  the  predicted  behi’.dor  of  both  a 


and  B  with  all-new  aaaembliee  ia  e&uae  to  euspect  that  Model  A  might  than 
do  aa  poorly  a*  Model  B  when  moatly  repaired  aaaembllea  are  provided  It. 
A  very  likely  conaequence  of  the  obaerved  trend  of  performance  la  that 


fleet  uaera  will  probably,  quietly  reduce  their  level  of  tank  uae  to  one 


allowing  more  relaxed  maintenance  aupport. 


Montha  After  laiue 


Figure  17- -Tanka  Down  for  Engine  and/or  Tranamieaion 
Replacement  at  130  Mile*  per  Month 
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fig,  18 — Equipment  Availability  is  only  a  partial  measure  of  materiel  a 

readiness  lor  combat.  In  general  availability  can  always  be  increased 
by  decreasing  the  rate  of  use.  Such  a  phenomenon  seema  to  occur  very 
often.  A  matter  of  equal  importance  to  readiness  is  the  performance 
to  be  expected  from  any  equipment  that  ie  available.  Vast  segments  ot . 
the  US  Army 's.;inventory  face  a  very  severe  dual  requirement.  Such 
equipment  ie  used  in  extensive  peacetime  training  programs.  The  equip* 
ment  has  to  be  available  not  only  for  training  but  also  for  any  emergency 
deployment  to  combat,  In  order  to  survive  if  combat  should  arise,  it  is 
necessary  that  the  equipment  continues  to  possess  an  adequate  of  residual 
or  combat  life,  The  preceding  examples  from  tank  life  give  sufficient 
evidence  of  a  probable  loss  of  residual  life  as  the  equipment  is  used  and 
ages.  The  data  of  mobility-affecting  parts  replacement  activity  were 
further  translated  into  a  measure  of  what  would  be  expected  in  tha  way 
of  tank  endurance  in  the  event  of  an  unexpected  50-mile  march  over 
hard-surfaced  roads.  In  order  to  score  a  success,  the  tanks  subjected 
to  this  hypothetical  test  must  be  available  to  start  such  a  march  and  then 
complete  it  without  a  mobility-affecting  deficiency.  The  march  was 
made  short  for  several  reasons.  The  principal  reason  was  that  over  ^ 

several  yaara,  the  observation  of  several  echedulad  marches  revealed 
that  t&nke  experienced  moat  of  any  deficiencee  to  about  100  miles  during 
the  first  50  of  those  miles.  Hence  a  success  to  50  miles  is  good  assur¬ 
ance  of  success  to  somewhat  more  than  100  miles,  Too  many  so-called  i 

readiness  teats  are  not  previously  unannounced,  On  announced  exer¬ 
cises  units  very  often  are  able  to  deploy  all  their  vehicles  initially, 

Were  a  deployment  requirement  to  be  isaued  at  some  other  time  the 
results  would  be  likely  to  be  far  different, 

For  the  hypothetical  march  measure,  the  mileage -incidence  of 
mobility-affecting  replacement  action*  was  assumed  to  follow  the  trend  « 

developed  from  the  main  body  of  tank  history  data,  In  the  particular 
chart  shown  the  training  requirement  was  taken  to  be  about  125  miles 
per  month.  Over  a  period  of  many  years  tanks  would  continue  to 
accumulate  mileages,  lose  availability,  and  do  less  well  against  ths  r 

unexpected  march.  The  10,  000-mile  line  ie  Just  one  rough  approxi¬ 
mation  of  what  an  equilibrium  tank  might  achieve.  The  march  model 
has  been  given  considerable  elaboration  with  study  devoted  to  the 

implications  of  various  functional  effects  for  different  utilization  * 

dependencies.  The  requirement  for  uncomplicated  visual  presenta¬ 
tion  has  led  to  the  reduction  of  all  results  to  formats  very  almilar  to 
that  shown. 


Fig,  IS-  Effect*  of  Aga  on  Hypothetical  Performance 
Tank*  on  a  SO- Mil*  March 
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Fig.  19-  As  part  of  flic  conclusion  o:  tms  general  survey  of  RAC's  management 
assisting  activities  in  the  area  of  combat  vehicles,  it  is  appropriate  to 
reintroduce  consideration  of  parts  support  costs,  At  125  miles  per  tank 
per  month  the  yearly  Support  of  a  fleet  of  1000  tanks  of  the  model  studied 
requires  the  funds  shown  in  the  accompanying  table.  During  the  first  year 
the  fleet  consume*  about  SI,  5  million  worth  of  parts  and  assemblies.  By 
the  third  year  over  $10  million  worth  are  being  consumed.  Track  replace¬ 
ment  by  far  comprises  the  biggest  single  slice  of  the  total  bill.  In  the 
third  year  the  "other  parts"  account  for  only  about  16  percent  of  the  total 
cost,  but  they  involve  a  great  variety  of  different  kinds  and  numbers  of 
repair  parts. 


ANNUAL  EXPENSE-- 1000  TANKS 


Year 

Mileage 

Engine 

Transmission 

Track 

Other 

Sum 


1 

0-1500 
$  740,000 
170,000 
500,000 
460,000 
Si, 870, 000 


2 

1500-3000 
$2, 540,000 
430,000 
5,130, 000 
1,400, 000 
$9,500,000 


Figure  19 


3 

3000-4500 
S  3,  000,000 
1 830,000 
4,900,000 
1,  600,000 
$10,  330,000 


Fig.  20— Money  is  not  the  only  penalty  of  vehicular  old  age.  Even  though  parts 
are  fed  into  the  tank  fleet,  the  net  availability  of  tanks  drop.  In  reality 
tank  users  have  to  pay  more  and  get  lass  as  their  vehicles  accumulate 
mileage,  In  this  chart  both  the  parts  costs  and  unavailability  of  tanks 
like  those  studied  are  shown.  Out  of  a  force  of  1000  tanks,  the  equivalent 
of  more  than  an  entire  battalion  are  on  th*  average  unserviceable  and 
unavailable  during  the  second  and  third  yeare,  Compared  with  the  first 
year,  parts  costs  have  increased  about  five-fold  and  unavailability  about 
three-fold  be  the  second  and  third  years. 

Fig.  21 — Practically  all  the  preceding  results  may  be  categorized  as  by¬ 
products  of  the  general  analysis  leading  to  the  determination  of  taxget 
ages  for  the  effective  in-use  lives  of  vehicles,  Through  suitable  weighted 


Annual 

Parts  Costs  Per 
1000  T&rj’s, 
Millions  of  Dollars 


Fig.  20-THE  DOUBLE  PENALTY- 
MONEY  and  DOWNTIME 


Mil*i/T»nk/Month 
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combination  of  the  foregoing  kinda  of  information  schedules  of  lives  were, 
determined  for  the, tank  model  studied.  These  lives, are  best -represented 
by  a  curve, on  a  mile  age -calendar,  age  plane,  In  general  effective  in-use 
life  -depends  on,  rate,  of  utilization.  In  fact  the;,presented  curve  applies 
only  to  an  operation  spectrum  of  uniform  utilization >  The  general  accu¬ 
mulation  of  mileage  may  be  represented  rby a  path  belonging  to  a 'para-, 
metered  family  of  usages.  Each  use  family  will  result  in  a  different  life 
curve.  As  long  as  the  mileage  paths  of  a  given  use  family  do  not  inter¬ 
sect  one  another,  the  time  mileage  plane  consists  of  points  as  sociable 
with  at  most  a  single  utilization  curve,  and  a  single  effective  life  curve 
can  be  constructed  uniquely.  Existing  history  data  do  not  provide  an 
adequate  basis  for  the  construction  of  a  realistic  life  model  employing 
intersecting  mileage  paths  of  single  families  and  leading  to  life  differences 
at  the  points  of  intersection.  Such  questions  are  interesting  from  the  pro¬ 
gramming  and  function-theoretic  point  of  view,  but  they  remain  well 
beyond  current  capacities  for  empirical,  experimental  resolution. 

Two  separate  effective  in-use  life  curves  are  shown  in  the  figure. 

The  one  applies  to  entire  tanks  and  the  other  to  separately  defined  tank 
mobility  systems.  In  general  higher  utilization  rates  may  be  expected 
to  increase  mileage  lives  except  at  very  high  use.  However,  the  higher 
mileages  are  achieved  in  much  shorter  times.  Much  of  the  management 
problem  arises  because  the  training  program  results  in  large  numbers 
of  old  model  tanks  with  relatively  low  mileages  and  in  equally  large 
numbers  of  newer  model  tanks  with  much  higher  mileages.  It  becomes 
necessary  to  establish  tank  life  paths  through  the  inventory  in  such  a 
way  that  in  the  long  run  the  less  obsolescent  tanks  are  also  the  ones  with 
the  lower  mileages.  To  have  obsolete,  low-mileage  tanks  and  modern, 
over -used  ones  at  the  same  time  achieves  nothing  more  than  a  use-, 
readiness-,  budget-paradox. 

RAC  has  presented  a  rapid  survey  of  many  of  the  factors  considered 
in  a  well-integrated  program  of  assistance  to  the  US  Army  in  the  man¬ 
agement  of  its  combat  vehicles.  Time  and  space  do  not  permit  treatment 
of  all  factors,  nor  do  they  suffice  for  adequate  explanation  of  the  integra¬ 
tion  to  final  result. 

¥ 

RAC's  activities  in  this  area  represent  a  continuing  sequence  of 
alternating  empirical  and  theoretical  efforts.  Over  the  years  the  guid¬ 
ing  doctrine  has  been  to  provide  a  steady  output  of  information  of  general 
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and  epecific  utility  to  the  Army  in  the  management  of  it*  combat  vehicle*. 
tv.  4r./crrr..»<=j.  i...  tn  k*  rnna4<»*nt  uH tK  tViM  materiel  experience  of 

US  Army  troop*  in  combat  unit*  undergoing  actual  training  and  yet  remain¬ 
ing  constantly  responsible  for  preserving  a  combat  ready  posture.  Appli¬ 
cability  to  high  population  fleet*  and  not  eleganee  in  an  artifically  reduced 
inventory  ha*  been  a  test  to  be  satisfied  for  all  resulting  conclusions  and 
recommendations. 


APPLICATION  OF  STATISTICS  TO  EVALUATE  SWIVEL 
HOOK-TYPE  CROSS  CHAIN  FASTENERS  FOR 
MILITARY  APPLICATIONS  OF  TIRE  CHAINS 

Otto  H,  Pfeiffer 

U.  S.  Army  Tank- Automotive  Center 
Warren,  Michigan 


ABSTRACT .  The  test  was  conducted  according  to  a  developed  experi¬ 
mental  deeign  to  determine  the  utility  of  ewivel-hook  cross-chain  fastener* 
for  military  applications  of  tire  chains. 

Dual  tire  chain  assemblies  (mounted  on  M35A1  test  vehicle)  consisting 
of  standard  Military-type  aide  and  croas  chains  and  three  types  (standard 
and  two  swivel*)  of  cross  chain  fasteners  were  subjected  to  an  accelerated 
wear  test  of  425  miles  on  dry  concrete  road  surfaces,  The  experimental 
results  were  expressed  in  terms  of:  (1)  Miles  to  failure  for  an  Individual 
croas  chain,  (2)  Weight  losses  oi  selected  cross  chains  and  (3)  Replace¬ 
ment  times  for  each  of  three  types  of  fasteners,  The  principal  response, 

(1)  miles  to  failure,  was  considered  to  be  exponentially  distributed; 
therefore,  logarithms  of  miles  to  failure  were  analysed  in  accordance 
with  the  structure  pf  the  experimental  design. 

Croas  chains  connected  with  swivel-type  fasteners  remained  func- 
tionable  about  twice  aa  long  as  the  crose  chains  connected  with  the 
standard-type  fasteners.  Both  swivel-type  fasteners  permitted  signifi¬ 
cantly  faster  cross-chain  replacement  than  the  standard  type,  although 
one  swivel  type  was  also  significantly  faster  to  manipulate  than  the  other 
sv/ivel  type, 

The  statistical  analysis  of  the  experimental  result*  indicated  the 
swivel  hook-type  croaa  chain  fasteners  used  in  this  test  resulted  in  a 
significant  increase  of  cross  chain  life  as  well  ae  simplification  of 
replacement. 

INTRODUCTION,  It  became  necessary  for  the  government  to  make 
a  decision  whether  to  consider  swivel  hook-type  cross  chain  fasteners 
in  future  procurement  of  tire  chains  and  components.  Some  information 
on  swivel  hooks  was  available,  but  it  was  considered  inadequate  for  the 
basis  of  a  decision, 
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A  test  was  proposed  to  utilize  two  military  trucks  equipped  with  various 
types  of  tire  chains,  to  be  conducted  on  both  concrete  and  gravel  road 
surfaces  (in  a  two-to-one  proportion)  for  a  total  distance  of  approximately 
300  miles . 

Products  of  two  manufacturers  of  tire  chains  and  chain  components 
were  available  for  the  test.  Each  had  a  swivel  hook  of  a  particular  design 
which  they  were  interested  in  selling  to  the  Government. 

Since  a  large  proportion  of  the  Military's  wheeled  cargo  vehicles  fall 
within  the  2j  ton  payload  class  (see  Figure  1)  having  9.  00  x  20  size  tires, 
it  was  desirable  to  test  tire  chains  of  this  dimension, 

It  was  requested  that  representative  cross  chain  samples  of  all  test 
chain  assemblies  from  the  two  manufacturers  hereafter  referred  to  as 
"Code  B"  and  "Code  C"  and  a  standard  Military  item  referred  to  as 
"Code  A"  be  subjected  to  a  metallurgical  examination  to  determine: 
mechanical  propertie s ,  macro-etch  quality,  case-hardened  depth,  and 
cross-sectional  hardness, 

*■  * 

Two  complete  tire  chain  assemblies  of  Code  B  were  to  be  placed  on 
two  rear  dual  wheels  of  one  vehicle,  diagonally  opposite  to  two  Code  A 
chain  assemblies.  The  same  arrangement  utilizing  Code  C  and  Code  A 
was  to  be  adhered  to  on  a  second  test  vehicle. 

During  the  course  of  testing,  it  was  proposed  that  two  brake  panic - 
stops  per  mile  be  made  after  the  vehicle  had  attained  maximum  speed. 

Wheel  spining  on  take-off  was  also  requested. 

A  review  disclosed  a  test  [3]  had  previously  been  conducted  on 
tire  chains  by  the  Government.  This  test  compared  the  Code  B  and 
Code  A  chain  only.  A  similar  chain  arrangement  to  this  proposed  test 
was  used,  but  the  vehicle  was  driven  500  miles  over  various  terrains 
and  on  surfaces  ranging  from  marsh  and  swamp  to  concrete.  Although 
this  test  indicated  some  advantages  of  the  swivel  hooks  in  comparison 
with  the  standard  Military  chain,  the  test  did  not  adequately  establish 
the  quantitative  nature  of  the  advantages. 

Only  letters  of  recommendation  from  commercial  sources  were 
available  regarding  the  Code  C  chain, 
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The  similarity  between  the  newly  proposed  test  and  the  test  previously 
conducted  brought  up  the  question:  What  could  be  learned  with  this  test 
that  had  not  been  found  out  before  ?  The  only  answer  that  appeared  obvious 
was  --  Nothing.1  Further  study  of  the  situation  anticipated  many  problems 
if  the  proposed  test  arrangement  and  procedure  were  to  be  followed; 

1.  Differences  in  the  vehicles,  not  only  in  weight  but  manufactur¬ 
ing  variances  as  tire  sizes,  tracking  characteristics,  and  braking  charac¬ 
teristics,  to  mention  a  few. 

2.  Differences  in  the  tire  chains,  both  length  of  cross  chains 
as  well  as  the  cross -chain  wire  diameter  and  of  major  concern  --  the 
difference  in  metallurgy  of  the  cross  chain  steel. 

3.  The  difference  in  road  contact  surfaces  when  chains  are 
compared  on  different  wheels,  or  worse  --on  different  vehicles. 

4.  Difficulty  of  data  analysis  or  supportable  conclusions  if  the 

test  were  to  be  conducted  in  a  haphazard  manner  or  without  accounting 
for  known  variables.  v 

We  decided  a  specially-designed  plan  must  be  developed  which  would 
produce  usable  data.  The  services  of  Dr.  Emil  H.  Jebe,  a  Research 
Mathematician  from  the  University  of  Michigan's  Institute  of  Science  and 
Technology,  were  obtained.  He  became  an  inseparable  part  of  the  pro¬ 
ject  until  final  conclusions  were  reached. 

CHOICE  OF  RESPONSE  AND  EXPERIMENTAL  UNIT.  It  was 
apparent  that  several  responses  should  be  studied.  Of  primary  interest 
were: 


1.  Miles  to  failure  of  each  individual  cross  chain, 

2.  The  weight  loss  of  the  cross  chains  associated  with  each 
type  of  cross  chain  fastener, 

3.  The  time  needed  for  replacement  of  worn-out  or  broken 
cross  chains. 
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The  fi t m»jAr  problem  in  development  of  a  suitable  experimental 
deaign  of  plan  was  to  determine  the  unit  for  meaauring  reaponaea.  The 
four  rear  dual  wheel*  and  the  lnaide  and  outaide  tire  of  each  dual  wheel 
were  the  firat  kind*  of  unit*  conaidered,  However,  aince  a  aingle  tire 
chain  cover*  the  complete  dual  wheel  (reaulting  in  only  four  unite  being 
available  at  the  aame  time),  no  aatiafactory  deaign  could  be  baaed  on  a 
wheel  aa  the  unit  unleaa  1200  to  2000  mile*  could  be  driven,  Even  ueing 
a  aingle  tire  a*  the  unit  would  have  given  only  eight  unita  and  difference* 
between  outaide  and  inaide  tires  would  have  to  be  eliminated,  Thle  kind 
of  unit  would  also  require  an  extended  period  of  driving, 

Further  discussions  disclosed  that  the  government  was  considering 
at  this  time  only  the  utilization  of  swivel  hooka  aa  repair  itema  for  the 
ample  supply  of  standard  Code  A  chains  presently  in  stock, 

An  interesting  thought  occurred  ••  why  not  use  standard  Code  A  tire 
chains  with  Code  B  and  Code  C  swivel -hook  fasteners  inserted  aa  if  they 
were  repair  items?  This  would  eliminate  several  of  the  anticipated 
problems.  The  cross  chains  of  Code  A  or  standard  military  tire  chains 
were  considered  to  be  for  all  practical  purposes  of  uniform  aiae  and 
metallurgical  composition. 

It  suddenly  became  evident  that  with  thia  concept,  a  larga  number 
of  experimental  units  would  become  available  if  we  conaidered  each  indi- 
viduarqroas  chain  aa  the  unit  of  measurement.  A  dual  tire  chain  consists 
of  26  cross  chains  or  13  on  each  half,  A  total  of  104  units  became  avail¬ 
able  which  could  bo  uaed  for  one  run  of,  say  --  300  to  500  miles. 

Some  minor  problems  were  encountered  in  the  acceptance  of  this 
experimental  unit,  The  swivel  hook-type  fastener  could  not  be  inserted 
into  the  same  link  of  the  side  chain  as  the  standard  crimp  hook'  It 
required  an  adjacent  link  90°  out  of  phase,  This  problem  was  solved  by 
spacing  the  cross  chains  unevenly  around  the  tire  aa  shown  in  Figure  2, 
Since  three  types  of  cross  chain  fasteners  were  to  be  tested,  four  cross 
chains  were  fastened  with  each  type.  The  remaining  cross  chain  was 
left  fastened  with  the  standard  type  and  was  not  used  for  teat  data.  Using 
this  arrangement,  there  were  32  cross  chains  for  each  fastener-type, 
evenly  distributed  over  the  eight  tires  of  the  four  dual  wheels. 
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A  comoletelv  ranrinmW*^  arrangement  c£  the  ciu»»  chain  fasteners 
on  each  wheel  seemed  undesirable.  A  randomised  starting  order  followed 
by  a  systematic  order  was  suggested.  The  cluster  of  four  cross  chains 
connected  with  the  same  type  fastener  on  each  tirt  became  the  experi- 
mental  unit  with  eight  replications  over  the  set  of  wheels. 

Four  chain  assemblies  were  fabricated  according  to  prescribed 
procedure  and  mounted  on  the  test  vehicle.  To  accelerate  the  rate  of 
wear  and  thereby  reducing  driving  distance,  a  test  course  (eee  Figure  3) 
consisting  entirely  of  concrete  was  selected.  Provisions  were  made  to 
equalize  right  and  left  turns  necessary  during  test  driving, 


STRUCTURE  OF  THE  DESIGN.  With  the  experimental  unit  now 
clearly  defined,  it  was  possible  to  develop  the  complete  overall  plan  for 
data  analysis.  There  were  certain  obvious  sources  of  environmental 
variation  present  which  could  not  only  be  removed  but  estimated  for 
magnitude.  These  sources  are  listed  as; 

1,  Difference  between  front  and  rear  dual  wheels. 


2,  Difference  between  right  and  left  side  dual  wheels, 

3.  Outside  versus  inside  tires. 


4.  Interactions  of  these  effects  with  each  other. 

5.  Possible  interactions  of  the  treatments  (types  of  cross 
chain  fasteners)  with  these  positional  differences. 


Since  there  were  three  clusters  •  three  experimental  units  -  of 
four  chains  (each  cluster  with  different  types  of  fasteners)  on  each  tire 
the  plan  may  be  described  as  a  Randomised  Complete  Block  Design  in 
eight  replicates  considering  each  tire  as  a  block,  The  variation  among 
blocks  was  also  to  be  subdivided  in  the  manner  just  outlined. 


A  formal  structure  for  the  design  could  then  be  established.  The 
usual  textbook  model  for  a  Randomised  Complete  Design  would  be 
satisfactory  lor  preparing  an  analysis  of  variance  of  these  observed 
results,  providing  the  usual  assumptions  could  be  made.  One  of  these 
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assumptions  is  that  the  observations  are  independently  and  normally 
diot* i’uulcu  [l]  [10]  ,  The  response  oi  primary  concern  here  is  miles  to 
failure  of  an  individual  cross  chain,  Therefore,  the  teat  plan  may  well 
be  considered  a  "life  test"  or  a  "wear  test".  Considering  that  our 
observations  were  "miles  to  failure",  we  realized  they  would  not  likely 
be  well-described  by  the  normal  probability  distribution,  This  point 
is  well  demonstrated  in  Figure  4.  It  appears  therefore  that  the  expo- 
ential  distribution  may  be  regarded  as  an  acceptable  probability  model 
for  the  observed  miles  to  failure.  With  this  in  mind,  our  analyses  of 
these  failure  data  have  been  generally  guided  by  the  considerations  set 
forth  in  a  series  of  papers  by  B,  Epstein,  B,  Epstein  and  M.  Sobel,  and 
M.  Zelen  which  appeared  in  several  Mathematical  and  Statistic  Journals 
(See  reference  list). 

The  exponential  distribution  in  its  probability  density  form  usually 
expresses  the  random  variable  in  some  quantity  directly  related  to  time. 
In  our  case,  d  =  miles-to-failure  may  be  used  as  the  random  variable, 

In  this  form,  a  constant  uniform  failure  rate  is  assumed.  As  was 
indicated  previously,  we  do  not  have  a  uniform  situation  in  this  tire 
chain  test  since  there  are  a  number  of  sources  of  variation  present, 

The  parameter  6  appearing  in  the  probability  density  form  represents 
the  mean  time  before  failures  of  the  cross  chains,  Based  on  Zelen's 
work  [19]  ,  a  more  complex  model  (Figure  5)  was  written  for  this  0 
in  the  exponential  distribution. 

Another  view  may  be  taken  for  observations  following  exponential 
distribution,  The  procedure  established  here  suggests  taking  logh- 
rithms  of  the  observations,  in  this  case  miles-to-failure  for  each  cross 
chain,  and  then  carrying  out  an  analysis  of  variance  considering  neces¬ 
sary  assumptions,  The  functional  relation  between  mean  and  variance 
for  exponential  distribution,  that  is,  the  standard  deviation  equals  the 
mean  [2]  suggests  the  use  of  the  logarithmic  transformation  (see  Figure 
6).  This  approach  is  also  discussed  by  Zelen  in  his  paper  [19]  ,  He 
finds  the  technique  acceptable  against  possible  departures  from  the 
strict  exponential  distribution  form, 

The  analysis  of  variance  of  the  logarithms  was  prepared  on  the  basis 
of  the  model  and  will  be  discussed  later. 


exponential  distribution  model 
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The  teat  waa  originally  planned  to  run  for  300  to  500  miles.  At  300 
milea,  the  teat  waa  temporarily  stopped  to  assess  the  aituation  up  to  that 
time.  It  was  already  clear  that  there  were  large  differences  in  miles 
to  failure  for  types  B  and  C  versus  type  A  even  though  approximately 
l/3  of  the  original  cross  chains  had  not  yet  failed,  This  large  number 
of  unfailed  chains  would  have  created  considerable  difficulty  when  analye- 
ing  the  results,  Driving  was  continued  to  about  425  milea  before  being 
terminated  for  other  reasons,  At  that  time,  six  of  the  original  cross 
chains  remained.  These  were  all  equipped  with  type  B  fasteners  and  were 
positioned  as  follows;  One  on  each  of  two  wheels,  and  four  on  one  wheel, 
as  shown  in  Table  I,  It  was  necessary  to  estimate  data  values  for  these 
six  In  order  to  maintain  a  balanced  and  simple,  straight-forward  analysis 
of  the  results.  In  general,  procedures  derived  from  work  by  B.  Epstein 
[8]  were  used  for  estimating  the  missing  data,  Using  Epatein'e  formula 
(see  Figure  7)  we  obtained  estimates  for  the  missing  values  on  the  two 
•  ingle  tires.  In  the  first  case,  n  =  4,  r  »  3,  =  309,  8  and  d^  »  226.  6. 

Solving  for  §  we  obtained  the  value  of  123,25.  This  formula  assumes 
that  failures  occur  randomly  at  any  time  starting  from  aero  miles  of 
travel.  Since  failures  did  not  occur  for  aome  distance  of  travel,  we 
estimated  the  minimum  "guarantee"  distance  A  by  the  formula 
A  s  d^  -  *&/  ,  In  this  case,  £  *  195.  79,  Combining  £  and  “s  ,  we  obtained 

319.  04  as  the  proper  estimated  MTBF"£or  the  cell,  Using  the  estimated 
cell  mean,  we  estimated  the  cell  total  as  n  (estimated  MTBF)  =  4(319,  04)  ■ 
1276. 16.  We  already  knew  the  actual  miles  to  failure  of  three  of  the  cross 
chains  In  the  cell;  therefore,  subtracting  this  value  from  the  estimated 
total  left  433,1  as  the  estimated  missing  value. 

The  same  method  was  used  to  estimate  the  second  missing  value  at 
504.  3. 

No  failures  of  B  type  swivel  hook  fasteners  were  evidenced  on  the 
outside  tire  of  the  left  front  wheel.  This  situation  posed  a  real  problem, 
The  Epstein  formula  used  for  estimating  the  previous  two  missing  values 
requires  at  least  two  failures  in  a  cell  if  it  were  to  be  applied  directly. 

A  variety  of  methods  for  solving  this  problem  were  considered,  includ¬ 
ing  schemes  based  on  using  the  weights  of  the  unfailed  cross  chains  at 
termination  of  the  test  driving,  All  these  schemes  were  rejected  as 
unsuitable , 

“-■Mean  Time  Before  Failure 


MILES  MILES  MILES  MILES 
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MILES  TO  Ml  LOME  OP  ORIGINAL  CROSS  CHAINS 


EPSTEIN'S 


Design  of  Experiments 


513 


In  order  to  obtain  a  useable  solution,  the  Epstein  formula  was  reapplied 
to  the  whole  of  Type  B  fastener  data,  Utilizing  the  two  previously  esti¬ 
mated  missing  values,  the  parameters  now  became  n  =  32  and  r  »  28  in 

this  instance.  The  mathematics  will  not  be  described  here,  But  stating 

28 

briefly,  &  and  ^  were  estimated,  then  32  (A  +  ^ )  -  E  d^  yielded  an 


estimated  total  for  the  entire  missing  cell,  This  estimated  total  /4  pro¬ 
vided  the  estimated  MTBF  or  ^  for  the  cell.  Individual  values  were  then 
determined  by  proportionality  of  each  cross  chain  weight  to  mean  weight 
of  the  cell.  The  estimated  missing  values  for  this  cell  ranged  between 
563  and  572  miles,  These  values  appeared  to  bs  too  high,  giving  the 
impression  we  were  favoring  type  B,  Applying  the  standard  analysis  of 
variance  "missing  plot  procedure"  for  the  Randomised  Complete  Block 
Design  to  the  logarithms  of  the  miles  to  failure  data  [14]  [18]  provided 
a  mean  value  for  the  entire  cell  although  it  was  not  a  useable  value.  The 
estimated  value  baaed  on  averaging  the  data  available  wai  276  miles,  but 
it  was  known  these  cross  chains  had  already  traveled  over  400  miles, 

The  distance  traveled  at  termination  of  test,  424,  9  miles,  could  also  have 
been  assigned  to  each  unfailed  croes  chain  but  this  would  havs  been  unfavor¬ 
able  to  type  B,  Therefore,  value*  estimated  as  already  described  were 
used  and  they  appeared  to  yield  an  acceptable  solution, 


There  are  several  approaches  which  may  be  followed  in  considering 
the  estimation  of  the  effects  of  intsrest  in  this  experiment,  For  complete¬ 
ness,  three  methods  were  considered  and  the  differences  among  the 
methods  were  small  for  this  test  program,  The  methods  considered  were: 


1,  Calculating  the  appropriate  simple  averages  of  the  miles  to 
failure  data 


2,  Estimating  the  parameters  in  Zelen's  model  as  described 

above 


3,  Estimating  in  terms  of  averages  of  the  logarithms  of  miles 
to  failure. 

The  latter  of  the  three  methode  was  used  in  the  analysis  of  variance  and 
will  be  discussed  further, 


Logarithms  of  the  original  data,  mile*  to  failure,  and  the  anti -logs 
of  the  mean  logarithms  are  shown  in  Table  II, 


TABLE  II 
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Comparing  the  average  for  type  A  fastener  (inside  tires  only,  expressed 
in  anti-log  form  as  147.  3  miles  with  values  calculated  by  the  other  two 
methods  -  -  155,  8  miles  and  155,  9  miles,  respectively)  we  find  it  slightly 
less.  This  somewhat  lower  figure  is  the  result  of  the  non-linearity  of 
the  logarithmic  transformation.  This  anti-log  value  is  really  an  estimator 
of  a  median  value  rather  than  a  mean, 

Estimating  the  differences  among  the  types  of  cross  chain  fasteners, 

-  r  the  ratios  as,  say  --  B/A,  C/A  and  B/C  was  the  next  concern.  It 
•  not  a  simple  problem  and  considerable  study  was  devoted  to  finding 
a  r  jasonable  solution.  A  method  discussed  by  Zelen  [19]  [20]  for 
estimating  such  ratios  and  finding  confidence  limits  for  the  ratio  gave 
extremely  wide  limits  for  these  ratios  frpm  our  experimental  data. 

From  the  averages  of  "log  miles"  for  "Insides  Tires",  "Outside  Tiree" 
and  "Combined  Averages"  presented  in  a  previous  table,  the  fastener- 
type  differences  expressed  in  logarithms  wore  calculated  The  variances 
of  these  differences  were  estimated  directly  by  taking  2( Experimental 
Error  Mean  Square)  /r,  where  r  is  the  number  of  values  averaged  to 
form  a  mean  for  a  single  fastener  type.  This  was  based  on  the  theorem 
that  the  variance  of  a  difference  is  the  sum  of  the  variances  of  the  quanti¬ 
ties  used  to  form  the  difference.  The  standard  deviation  was  obtained 
from  the  variance  result  Just  stated.  Confidence  Intervals  were  then 
formed  by  taking  the  observed  mean  differences:  (B-A)  +  (etandard 

error)  where  t  in  this  case  was  t^  ■  1.  987.  The  k  ■  86  degrees 

of  freedom  comes  from  the  Pooled  Error  Mean  Square  determined  from 
the  analysis  of  variance  presented  later,  The  confidence  intervals  obtained 
are  for  differences  of  averages  expresssd  in  logarithms. 

There  was  a  considerable  difference  in  the  average  miles  to  failure 
(about  50  miles)  considering  all  the  fasteners  combined  between  the 
inside  Tires  and  the  Outside  Tires,  Consids ring  this  difference,  it 
was  decided  to  present  separate  results  for  Inside  and  Outside  Tires  and 
then  combined  averages,  as  seen  in  Tables  II  and  III,  Returning  results 
to  original  scale  of  miles  to  failure  was  desirable.  It  was  observed  that 
the  anti-log  of  the  difference  (B  -  A  on  Inside  Tires)  was  nearly  the  ratio 
of  miles  to  failure  for  B/A  as  given  by  the  data  in  Table  II.  Anti-logs 
taken  for  the  lower  and  upper  confidence  limits  for  the  differences  like¬ 
wise  became  approximate  confidence  limits  for  the  ratios,  These  results 
are  also  listed  in  Table  III. 
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Estimated  Type  Differences  In  Logarithms  snd  Estimated 
Ratios  of  Miles  to  Failure  by  Types  of  Crossbar  Fasten¬ 
ers  and  the  Associated  Confidence  intervals 


Description 

Estimate 

Lower 

Upper 

Difference  Ratio 

Limit 

Limit 

Inside  Tlreo 

B-A 

0.240 6 

0.1289 

0. 3523 

D/A 

1.7400 

1.3460 

2.2510 

C-A 

0.2292 

0.1175 

0.3409 

C/A 

1.6950 

1.3110 

2.1923 

B-C 

0.0114 

-0.1003 

0,1231 

B/C 

1.0260 

0,7938 

1.3280 

Outside  Tires 


B-A 

B/A 

0. 3363 
2.1690 

0.2246 

1.6770 

C.4480 

2.O050 

C-A 

C/A 

0.1490 

1.4090 

0.0373 

1.0900 

Q.2607 

1.8230 

B-C 

B/C 

0.1873 

1.5390 

0.0756 

1.1900 

0.2990 

1.9910 

Combined 

B-A 

B/A 

0.2864 

1.9430 

0.2095 

1.6200 

0.3673 

2.3300 

C-A 

C/A 

0.1091 

1.5460 

C. 1102 
1.2690 

0.2680 

1.6540 

B-C 

B/C 

0.0993 

1.2560 

0.0204 

1.0460 

0.1782 

1.5070 

TABLE  1 1 1 
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It  is  to  be  noted  that  these  confidence  limits,  when  expressed  in  the  original 
scale  in  miles,  are  really  confidence  limits  for  a  median  mileage  ratio 
figure, 

ANALYSIS  OF  TOTAL  VARIATION.  The  analysis  of  variance  in  terms 
of  logarithms  of  miles  to  failure  is  presented  in  Table  IV, 

The  selected  chain  arrangement  as  previously  described  was  a  cluster 
of  four  cross  chains  for  each  type  of  fastener  s yetematically  arranged 
around  each  tire.  When  comparing  error,  it  was  found  the  error  mean 
square  (cased  on  the  clusters)  was  about  equal  to  the  mean  square  for  the 
cross  chains  within  the  clusters  (except  on  the  inside  tires  where  there 
was  some  difference,  but  in  the  wrong  direction).  It  was  decided  to  calcu¬ 
late  the  pooled  error. 

The  results  in  Table  IV  bear  out  the  large  differences  between  the 
types  of  fasteners  already  displayed  in  Tables  II  and  III.  Other  points  to 
be  noted  from  Table  IV  are; 

1,  Left  side  versus  right  tide  effect  is  small  (about  equal  to 

error), 


2.  The  front  wheels  versus  rear  wheels  effect  is  large  in  rela¬ 
tion  to  error  although  this  effect  is  mostly  associated  with  the  outside 
tires. 


3.  Individual  wheels  differ  considerably  from  what  might  be 
expected  if  predictions  were  based  only  on  the  left  versus  right  and  front 
versus  rear  effects,  This  effect  it  shown  by  the  interaction  line  which 
is  again  largest  for  the  outside  tires. 

4.  The  difference  in  miles  to  failure  for  outside  tires  versus 
inside  tires,  about  40  miles,  does  not  appear  to  be  a  chance  effect, 

Most  of  this  difference  was  associated  with  the  left  front  wheel,  however, 

5.  During  the  detailed  examination  there  was  some  question 
regarding  the  uniformity  or  behavior  of  the  types  of  fasteners  on  the  out¬ 
side  tires  versus  inside  tires,  Although  not  shown  in  Table  IV,  the  effect 
of  an  interaction  of  types  by  outside  versus  inside  was  calculated  (removed 


Design  of  Experiments 


519 


from  error  with  14  decree*  of  freedom).  Ar.  F  i'itlu  of  about  2.  4b  was 
obtained  when  comparing  the  mean  equate  obtained  with  pooled  error  mean 
equate,  The  probability  of  such  a  value  of  more  extreme  occurring  by 
chance  was  about  .  09. 

OTHER  TESTS,  During  the  temporary  termination  of  the  test  at  300 
miles,  a  Median  Test  described  in  works  by  Mood  [13]  was  applied  to  the 
data,  From  this  test  at  the  300  mile  level,  it  was  reasonable  to  conclude 
that  B  and  C  type  fasteners  were  much  superior  to  type  A;  however,  it 
seemed  desirable  to  perform  additional  test  driving  to  further  quantify  the 
experimental  results. 

OTHER  RESULTS  OF  INTEREST, 


1.  Weight  loss; 

Weight  loss  measurements  for  Individual  cross  chains  were  made 
at  20-mile  intervals  during  the  test  program  while  original  chains  remained 
intact.  To  remove  correlation  between  successive  weighing!,  different 
cross  chains  were  aelected  for  weighing  at  the  end  of  each  20-mile  interval, 
At  each  weighing,  one  cross  chain  for  each  type  of  fastener  was  removed 
from  each  wheel  and  then  replaced  in  its  original  position.  With  the 
limited  numbsr  of  cross  chains  available,  it  was  necessary  to  repeat 
weighting  at  the  end  of  180  miles.  The  resulting  weight  loss  data  was 
plotted  against  distance  driven  (see  Figure  8).  These  weight  loss  data 
show  that  all  cross  chains  (regardless  of  fastener  type)  tended  to  loss 
weight  at  approximately  the  same  rats,  Considering  this  result,  the  longer 
life  of  cross  chains  associated  with  the  B  and  C  type  fasteners  must  be  due 
to  a  spreading  of  the  wear  over  the  entire  surface  of  the  cross  chsin  pro¬ 
duced  by  the  rotational  motion  permitted  by  the  ewivel-typa  fasteners. 

Many  of  the  type  C  swivel  fasteners  had  a  forge  fiashir^  on  the  hookjihank 
(shown  in  Figure  9)  which  restricted  the  rotational  motion,  Depending 
on  whether  there  were  two,  one ,  or  none  of  these  hooks  with  "flash",  a 
particular  cross  chain  might  rotate  freely,  only  partially  (wind-up) ,  or 
not  at  all,  Such  results  could  sccount  for  the  observed  difference  in  life 
of  the  B  and  C  types.  Figures  10,  11,  and  12  show  specific  wear  pat¬ 
terns  associated  with  each  type  of  fastener.  The  curved  wear  pattern 
established  in  the  C  type  is  assumed  to  be  the  result  of  one  non-rotating 
hook  causing  chain  wind-up. 
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2,  Replacement  Time: 

At  each  replacement  of  a  failed  cross  chain  and  also  during  the 
weighing  procedure  in  the  shop,  the  time  required  for  removal  and  rein¬ 
stallation  of  the  cross  chain  was  recorded,  A  separate  record  was  kept 
for  shop  work  and  field  work.  There  was  some  difference  associated  with 
the  work  site;  however,  the  most  pronounced  difference  was  associated 
with  type  of  fastener,  An  analysis  of  variance  of  these  differences  could 
have  been  calculated,  but  the  mean  differences  in  observed  time  between 
the  three  types  were  so  large  that  a  detailed  analysis  did  not  seem  to  be 
needed,  The  data  were  analysed  using  the  Wilcoxon-Mann-Whitney  Statistic 
(ranking  method)  [12]  [15]  .  There  wai  found  to  be  a  significant  difference 
between  the  averages  in  all  cases.  The  average  replacement  time  and 
re-installation  time  and  ratios  are  tabulated  In  Table- V, 

All  removals  and  installations  of  the  A-type  fastener  were  done  with 
a  special  tool  (see  Figure  13)  provided  for  this  test.  An  unsuccessful 
attempt  was  made  to  remove  an  A-type  fastener  with  the  toola  provided 
in  the  standard  tool  kit  of  the  vehicle, 

CONCLUSIONS.  It  is  clear  from  the  results  obtained  that  the  crose 
chain  fasteners  type  B  and  C  (ewivel  hook)  are  euperior  to  the  etandard 
type  fastener.  Thie  superiority  is  primarily  described  by  comparing 
average  miles  to  failure  or  by  the  ratios  of  average  miles  to  failure. 

From  these  ratios  for  inside  tires  only,  we  observe  the  swivel-hook 
fasteners  to  be  about  70  percent  better  on  the  average.  On  the  outeide 
tires,  we  find  type  B  about  117  percent  better  than  type  A,  and  type  C  55 
percent  better  than  type  A.  Confidence  limits  for  the  true'  ratios  of 
superiority  show  a  minimum  of  at  least  30  percent  improvement  on 
inside  tires,  and  possibly  as  much  ae  120  percent  for  the  ewivel  hooks 
when  expressed  in  terms  of  two-sided  95  percent  confidence  intervale, 
These  results  were  far  less  uniform  on  the  outside  tires, 

The  next  question  raised  is,  "Are  the  type  B  ewivel  hooks  better 
than  the  type  C  7"  The  observed  difference  on  the  inside  tires  is  small. 

On  the  outside  tires  the  data  indicate  a  significant  difference  between 
type  B  and  C.  A  large  part  of  the  superiority  of  B  and  C  is  found  on  the 
left  front  outside  tire.  Type  B  is  also  better  than  C  on  the  other  three 
outside  tires,  but  to  a  varying  degree, 
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Special  Chain  Tool  with  Code  A  Fastener  in  "Ready  to  Close"  position. 

FIGURE  13 
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As  described  before,  it  was  noted  the  type  B  fastener  allowed  better 
cross-chain  rotation  than  the  type  C  so  that  part  of  the  difference  between 
B  and  C  may  be  ascribed  to  this  characteristic,  although  the  siseable 
difference  in  performance  for  outside  and  inside  tires  it  baffling. 

The  replacement  times  are  highly  favorable  to  the  swivel  hooks 
although  type  C  was  found  to  be  somewhat  better  than  B,  Thus,  it  seems 
that  minor  modifications  might  make  the  two  swivel  hooks  about  equal  in 
performance  and  replacement  time, 

A  field  trial  conducted  using  chains  completely  assembled  with 
swivel  hooks  would  be  worthwhile  to  determine  the  extrapolation  factor 
for  normal  field  conditions  from  the  accelerated  test  conditions. 

When  considering  the  use  of  the  swivel-hook  type  fasteners  as  replace 
ment8  in  military  tire  chains,  it  appears  from  the  data  obtained  that  the 
present  experiment  has  been  adequate. 
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ERROR  ANALYSIS  PROBLEMS 
IN  THE  ESTIMATION  OF  SPECTRA 

Virginia  B.  Tipton 

White  Sande  Mitaile  Range,  New  Mexico 


ABSTRACT .  Power  apectral  denaity  function*  are  eatimated 
digitally  by  evaluating  the  Fourier  cosine  transform  of  the  autocorrela¬ 
tion  function,  In  order  to  obtain  reliable  averages  with  which  to  describe 
the  autocorrelation  function  it  is  necessary  to  limit  the  resolution  with 
which  it,  and  its  transform,  can  be  described,  Is  it  possible  to  evaluate, 
or  to  express  analytically,  the  accuracy  with  which  the  computed  spectrum 
represents  the  true  spectral  density  function? 

1NT RODUCTION.  The  use  of  power  apectral  density  functions  to 
describe  the  frequency  content  of  a  time  function  has  been  a  common 
engineering  practice  for  some  time,  developing  originally  from  the 
communications  engineers'  concern  with  separating  signals  from  noise 
in  transmission  systems.  At  the  same  time  the  statisticians'  approach 
to  the  study  of  random  fluctuations  in  time  series  data  led  to  the  develop¬ 
ment  of  autocorrelation  functions  as  a  descriptive  tool,  The  bridge  bet¬ 
ween  these  two  approaches  to  the  study  of  noise,  which  is  simply  high 
frequency  random  variations  superimposed  on  the  desired  data,  was  the 
discovery  of  the  now  well-known  Wiener-Khinchin  relationship,  This 
relationship  simply  states  that,  except  for  a  constant  factor,  the  power 
spectral  density  function  and  the  autocorrelation  function  of  a  stationary 
random  process  are  a  Fourier  transform  pair.  Since  the  autocorrelation 
function  is  an  even  function  of  its  time  lag  r,  the  complex  Fourier  trans¬ 
formation  process  simplifies  to  a  real  cosine  transformation  which  can 
easily  be  carried  out  by  a  digital  computer. 

The  digital  computation  of  power  spectral  density  functions  is  be¬ 
coming  an  increasingly  more  important  part  of  data  reduction  work,  It 
is  now  being  applied  experimentally  to  the  study  of  random  errors  in 
trajectory  measuring  instrumentation  systems,  as  well  as  to  the  more 
traditional  applications  in  vibration  data  analysis  and  telemetry  problems, 

However,  in  order  that  the  spectral  estimates  computed  may  be  of 
value  to  the  data  user,  we  must  be  able  to  describe  in  some  way  the 
reliability  with  which  the  computer  spectrum  approximate*  the  true 
spectral  density  function;  that  is,  we  must  be  able  with  some  degree  of 
confidence  to  place  limits  upon  the  errors  of  our  estimation. 
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THE  WStviK  DATA  k£uu^iiuw  SHECTRUM  ANALYSIS  PROGRAM, 

The  derivation  of  the  computer  programming  equations  used  at  WSMR  Data 
Reduction  Directorate  was  first  given  in  a  report  written  in  1957,  "The 
Digital  Computation  of  Power  Spectra,  "  by  L.  M,  Spetner  of  Johns  Hopkins 
University,  This  digital  process  is  baeed  on  the  Wiener-Khinchin  relation¬ 
ship;  that  is,  it  first  computes  the  autocorrelation  function  of  the  random 
data  and  then  determines  its  Fourier  transform,  which  is  the  power 
spectral  density  function.  In  order  to  separate  the  noise  data  from  any 
constant  ( zero  frequency)  component,  the  input  data  are  firet  averaged 
and  then  this  data  mean  is  subtracted  from  the  original  data.  This  pro¬ 
cess  insures  that  the  average  of  the  residuals  will  be  zero,  a  condition 
which  must  be  met  if  the  Fourier  transform  is  to  axiat.  In  order  to 
■..iminate  any  linear  trend,  or  a  quadratic,  a  least  squares  2nd  degree 
curve  is  then  fit  and  removed.  We  are  now  ready  to  compute  the  auto¬ 
correlation  function  of  the  residuals,  which  we  assume  then  to  be  both 
random  and  stationary, 

At  this  point  it  is  well  to  say  a  few  words  about  random  processes  In 
general,  A  random  process  is  a  collection,  or  ensemble,  of  time  func¬ 
tions  such  that  the  ensemble  can  be  characterized  by  its  statistical  proper¬ 
ties,  In  studying  noise  problems  we  are  usually  not  overly  concerned  with 
that  individual  time  function  which  we  happened  to  observe,  since  any  of 
the  member  functions  of  the  ensemble  could  have  occurred  with  equal 
probability,  Rather  we  are  interested  in  determining  from  the  observed 
function  the  statistical  properties  which  charactsrizs  the  entire  ensemble, 
For  a  special  class  of  random  processes  (that  is,  for  those  which  are 
both  stationary  and  ergodic)  this  can  be  ms  because  it  has  been  shown 
(elsewhefc)  that  in  such  cases  the  process  averages  across  the  ensemble 
are  equal  to  the  time  averages  along  a  aingls  rapre sentatlve .function  from 
the  ensemble  (See  Figure  1). 

The  autocorrelation  function  for  a  random  process  Is  defined  as  the 
ensemble  average  of  t.te  product  of  each  function  timet  itself  shifted  by 
a  time  delay  t  , 

(1)  R(t)  =»  f(t)  ■  f(t  +  t)  , 


where  the  wavy  bar  indicates  averaging  across  the  ensemble, 
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If  we  are  dealing  with  a  single  random  function  from  an  ergodic 

.  ..  _  1.  1  .  - M  M  \  f-.  —  ^  /•  a  ««al  sti  nn  A  r*  m  A  •  A  t  ^  m  A 

CUOCiUUlC  )  G  Ui«*  biWit  *  h.**w  w»  ►  W  « - -  •  - - - - - -  - - - “ - -  --  *- 

average  over  the  function 

T 

w  »w  =  it  r  yo  <(«)«• + ^ 

In  the  digital  case  where  the  integral  is  replaced  by  a  Bummation 
over  the  range  of  data  points  N  and  time  delay  m,  this  becomes 

N-m 

(3)  R(m)  =  -r—  £  f(i)  f(i  +  m)  . 

N-m  iml 


This  autocorrelation  function  has  several  interesting  properties: 

(1)  It  is  an  even  function,  i,  e,  ,  R(-  t)  *  R(t)  (a  property 
which  is  useful  in  determining  its  Fourier  transform,  ) 

(2)  The  value  of  R(r)  for  t  *  0  equals  the  average  power  of 
f(t),  or  in  statistician's  language,  the  variance  of  the  function, 

(3)  The  value  of  R(t)  is  bounded  by  its  value  at  t  =0,  so 

that  the  computed  autocorrelation  coefficients  can  easily  be  normalised 
to  give  unity  autocorrelation  for  sero  time  delay, 

If  the  function  is  truly  random  then  its  autocorrelation  function 
will  rapidly  approach  zero,  since  the  values  of  f(t  +r),  as  t  increases, 
are  not  dependent  upon  the  value  of  f(t).  Thus  a  typical  normalized 
autocorrelation  curve  of  a  random  noise  record  will  have  the  shape 
indicated  in  Figure  2. 

However,  it  should  also  be  pointed  out  that  the  converse  is  not 
so  -  it  cannot  be  shown  that  because  the  autocorrelation  function 
approaches  zero  as  t  increases  thatthe  given  function  is  necessarily 
random, 

Once  the  autocorrelation  function  has  been  found,  the  power 
spectral  density  function  is  computed  from  it  by  taking  its  Fourier 
transform 
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4>(w)  = 


,  -iwr 
R(t  )e  dr 


T  =  -« 


Using  the  property  that  R(-t)  =  R(t),  this  becomes 


(5) 


4>(w)  =  2  \  R(t)  cos  ur  dT  , 


The  spectrum  is  estimated  for  discrete  values  of  u  ■  —  ,  where 

m 

K  is  an  index  ranging  from  0  to  m,  and  m  the  number  of  autocorrelation 
coefficients  computed. 

The  resulting  estimates  are  smoothed  using  a  3-point  symmetric 
filter,  with  weights  (0.  23,  0,  54,  0. 23)  and  plotted  as  function  of  frequency. 

(For  reference,  the  digital  computing  formulas  used  in  the  program 
will  be  listed  as  an  appendix  to  the  paper. ) 

THE  PROBLEM  OF  ERROR  ANALYSIS,  The  problem  confronting 
us  now  is  chiefly  this*  How  can  we  express  the  errors  Involved  in 
estimating  spectra  by  this  digital  process?  Or  in  other  words,  with 
what  confidence  can  we  say  that  the  spectrum  we  have  computed  repre¬ 
sents  the  true  spectrum  of  the  process  we  are  studying?  Can  we  put 
limits  on  our  error,  perhaps  in  tho  form  of  a  statement  such  as,  "our 
estimate  is  within  +_5 %  of  true  spectrum"  and  have  perhaps  90  or  95% 
confidence  that  we  are  right? 

The  problem  appears  to  be  in  balancing  the  frequency  resolution 
we  can  achieve,  that  is,  the  number  of  points  used  to  estimate  the 
spectral  curve,  against  the  reliability  with  which  they  are  computed. 

The  maximum  frequency  resolution  (Af)  in  our  digital  process  is  deter¬ 
mined  by  the  highest  frequency  we  can  distinguish  in  the  data  (*mftx)  and 

the  number  of  time  delay  averages  (M)  for  which  we  computed  the  auto¬ 
correlation  function. 
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But  the  highest  distinguishable  frequency  is  limited  by  the  rate  at 
w  hich  the  original  data.eamples  were  digitised.  The  sampling  rate  as 
given  in  Spetner's  equations  must  be  at  least  twice  the  highest 
frequency  present. 

By  the  time  the  data  arrive  in  digital  form  at  the  Data  Reduction 
Computer  facility  we  no  longer  have  any  control  over  the  digitizing 
rate  (l/At)  or  the  length  of  the  data  sample  [T(seconds)  =  (N  points)  ■ 

(At  seconds)}  ,  We  must  assume  that  the  data  users  chose  a  sampling 
rate  high  enough  to  minimize  aliasing  errors,  that  is,  the  folding  back 
of  frequencies  higher  than  £  so  that  they  appear  as  some  sub-multiple 
of  themselves  in  the  frequency  range  we  can  observe, 

In  addition,  the  number  of  time  delay  averages  used  to  describe  the 
autocorrelation  function  is  limited  by  the  length  of  the  data  sample.  In 
practice,  we  generally  limit  M  to  approximately  one -tenth  of  the  number 
of  data  points  N.  (M  =  N/10. )  We  could  increase  the  number  of  time 
delay  averages  computed,  but  only  at  the  cost  of  reliability  of  them.  As 
M  increases  the  number  of  data  points  available  to  average  decreases. 
Thus  this  could  not  solve  our  problem,  and  at  present,  no  other  eolution 
hae  been  found, 
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VALIDATION  PROBLEMS  OF  AN 
INTERFERENCE  PREDICTION  MODEL 

William  B.  McIntosh 
U,  S,  Army  Electronics  Proving  Ground 

The  Electromagnetic  Environmental  Test  Facility  (EMETF)  of  the  US 
Army  Electronic  Proving  Ground,  Fort  Huachuca,  Arizona,  is  being  devel¬ 
oped  to  give  solutions  to  a  host  of  communications -electronic  problems, 
most  of  which  in  one  way  or  another  arise  from  the  fact  that  military  demands 
imposed  upon  the  electromagnetic  spectrum  require  vastly  more  space  in 
the  spectrum  than  is  available  for  this  purpose.  A  consequence  of  the 
resulting  crowding  is  interference  between  electromagnetic  equipments. 

The  EMETF  is  designed  to  provide  experimental  data  bearing  on  the  inter¬ 
ference  problem  in  its  broadest  sense,  A  subsidiary  and  included  feature 
is  provision  of  data  on  the  ability  of  communications-electronic  systems 
to  perform  intended  missions  in  the  absence  of  competing  electromagnetic 
signals. 

Although  the  ultimate  EMETF  will  be  a  facility  capable  of  providing 
these  answers  for  communications  equipment,  for  radar,  for  navigation 
devices,  for  data  transmission  links,  and  in  fact,  for  all  army  electronic 
activities,  this  presentation  will  be  confined  to  aspects  of  voice  communi¬ 
cation  by  radio. 

Several  years  ago,  the  EMETF  was  conceived  of  as  primarily  a  huge 
outdoor  field  test  facility,  spread  over  some  2400  square  miles,  In  the 
District  of  Columbia  area,  this  facility  would  have  stretched  from  one  side 
of  Washington  to  the  other  side  of  Baltimore.  An  artist's  concept  of  the 
field  facility  is  shown  in  figure  1.  Originally,  24  transmitter  sites  and 
two  transmitte r- receiver  siteswere  deployed;  the  latter  two  sites  were  the 
test  sites.  At  each  of  the  transmitter  sites  equipment  was  grouped  around 
a  control  van,  The  master  control  center  is  located  at  one  of  the  test 
sites.  From  the  center,  one  or  more  transmitters  at  any  one  or  more  van 
sites  may  be  controlled  for  test  purpose. 

The  basic  test  unit  is  a  cycle,  figure  2,  which  requires  30  seconds; 
it  consists  of  energizing  the  desired  transmitter,  and  recording  the  test 
link  performance,  as  will  be  described  later,  Then  the  entire  environ¬ 
ment  is  turned  on,  and  the  link  performance  again  measured,  If 
degradation  has  occurred,  the  next  step  -  and  a  long  one  -  consists  of 
a  search  for  the  one  or  more  transmitters  responsible  for  the  degradation, 
These  operations  produce  the  basic  field  data, 
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The  concept  of  test,  and  with  it  the  field  facility,  has  evolved  to  the 
point  where  now  the  field  facility  has  been  reduced  in  size,  and  the  major 
share  of  useful  output  is  being  derived  from  a  computer  simulation  program 
known  as  the  Interference  Prediction  Model  (IPM).  The  IPM  requires  a 
considerable  amount  of  input  data  of  various  sorts,  which  has  lead  to  the 
creation  of  a  third  unit,  die  Instrumentation  Workshop. 

The  ultimate  form  of  the  IPM,  as  it  is  currently  envisioned,  is  shown 
in  figure  3. 

The  model  requires  input  data  specific  to  the  equipment  or  concept 
under  test  as  well  as  a  specific  description  of  the  problem.  Internally, 
the  model  consists  basically  of  five  modules.  Figure  3  also  shows  in 
block  diagram  form  how  each  module  is  developed  and  validated. 

The  propagation  module  performs  one  basic  function.  It  describes 
the  attenuation  which  is  expected  as  an  electromagnetic  wave  front  travels 
between  a  transmitter  and  a  receiver. 

The  equipment  module  incorporates  the  necessary  equations  which 
describe  what  happens  to  electrical  signals  as  they  pass  through  the  equip¬ 
ment.  The  scoring  module  translates  the  processed  signal  at  the  receiver 
output  into  a  measure  of  how  much  of  the  original  intelligence  remains. 

The  tactical  deployments  module  incorporates  several  preselected  deploy¬ 
ments,  each  containing  the  physical  location,  in  three  dimensions,  of 
every  piece  of  emitting  or  receiving  equipment,  plus  information  on  the 
topography  in  the  form  of  an  XYZ  matrix,  in  meters,  at  500  meter  inter¬ 
vals  in  the  XY  plane.  The  radio  frequency  assignment  also  becomes  a  part 
of  this  module. 

The  statistical  module  is  at  present  largely  undeveloped.  In  time, 
however,  it  will  be  used  to  convert  an  essentially  deterministic  model  into 
a  stochastic  one.  The  essential  purpose  of  this  paper  is  to  describe 
certain  problems  which  have  been  encountered  in  an  attempt  to  provide 
an  interim  stochastic  capability  by  different  methods. 

I  will  next  outline  a  test  problem  in  which  the  important  outputs  will 
be  obtained  from  the  IPM,  but  for  which  actual  hardware  of  the  test  item 
is  available  for  certain  measurements. 
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The  preparation  consists  of  establishing  the  location  of  all  equipments, 
both  of  the  test  type  and  other  types  which  will  share  the  environment,  the 
assignment  of  radio  frequencies ,  the  inputs  of  the  equipment  characteristic 
and  the  like.  From  the  entire  collection  of  communication  bets  in  the 
chosen  deployment,  a  sort  of  stratified  random  sample  is  chosen  for  test. 
Stratification  is  based  upon  the  frequency  of  different  type  nets  as  well  as 
on  the  relative  importance  of  net  type  to  mission  success.  Thus,  if  there 
is  only  one  command  net  from  Corps  to  Division,  that  net  is  included. 

From  the  many  command  nets  between,  say,  infantry  platoons  ?.nd  squads, 
several  are  chosen  at  random. 

In  the  present  use  of  the  model,  one  basic  question  is  asked.  What, 
on  some  scale,  is  the  overall  system  effectiveness?  This  question  is  often 
asked  for  a  standard  system  and  for  a  proposed  replacement,  whereupon 
the  comparison  will  provide  useful  information  to  those  who  make  procure¬ 
ment  decisions.  The  systems  effectiveness  measure  now  in  use  depends 
upon  a  somewhat  involved  procedure  which  results  in  every  test  link  being 
classified  as  providing  or  not  providing  acceptable  performance.  Then 
the  effectiveness  measure  is  merely  an  index,  being  the  ratio  of  accept¬ 
able  links  to  the  total  tested.  Since  the  initial  measure,  intelligibility, 
is  changed  from  a  continuous  variate  to  a  binomial,  many  links  which  have 
been  measured  inaccurately  will  still  be  classified  correctly.  Further, 
to  the  extent  that  the  model  is  imprecise  but  lacking  in  bias,  errors  of 
classification  in  one  direction  will  tend  to  be  balanced  by  other  errors  in 
the  opposite  direction.  Thus,  for  the  index,  we  really  do  not  need  to  be 
concerned  in  great  detail  with  the  goodness  of  our  answer  for  each  link. 

But  people:  do  ask  such  questions.  And  ultimately  we  would  like  to 
answer  such  queries  as;  How  well  can  some  specific  platoon  communi¬ 
cate  with  its  company  headquarters?  We  no  longer  will  be  satisfied  with 
knowing  how  well  on  the  average  a  platoon  can  communicate  with  a 
company,  nor  will  we  accept  a  simple  yes  -  no  answer. 

Given  this  ultimate  desire  to  answer  questions  about  any  communica¬ 
tions  link,  of  necessity  we  must  accept  a  stochastic  answer.  Even  if  we 
have  developed  a  perfect  model,  in  the  technical  sense, this  will  be  true. 
Communications  equipment  will  continue  to  exhibit  interunit  variability, 
operators  will  not  all  have  identical  hearing  ability  or  training;  and  most 
of  all,  propagation  loss  will  continue  to  be  an  important  variate.  Even 
if  we  should  come  to  know  the  form  and  moments  of  all  pertinent  distri¬ 
butions  of  equipment  characteristics,  we  won't  have  any  way  of  knowing 
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the  individual  characteristics  of  the  specific  equipment  at  any  given  geo¬ 
graphic  point.  Anyway,  in  reality,  those  equipments  may  be  a  few  to  a 
few  hundred  meters  at  least  away  from  where  we  have  them  located  in  the 
problem,  Even  though  we  learn  all  there  is  to  know  about  the  effects  of 
atmospheric  conditions,  terrain,  and  vegetation  on  path  loss,  we  can't  know 
what  the  precise  atmospheric  factors  would  be  if  the  tactical  operation  we 
are  simulating  were  to  exist.  Nor  can  we  precisely  describe  the  minutiae 
of  the  terrain  over  all  direct  paths  and  multipaths  between  all  necessary 
pairs  of  points  taken  from  a  vast  set. 

The  best  we  can  hope  for  --  and  this  it  seems  is  realistic  --  is  to  say 
with  some  chosen  confidence,  that  if  equipment  is  approximately  where  it 
is  supposed  to  be,  if  the  various  environmental  conditions  are  approximately 
those  used  as  model  inputs,  if_  we  have  studied  a  sufficiently  large  sample 
of  the  equipments  in  question  --  then  a  given  communication  link  will 
exhibit  a  performance  somewhere  between  points  A  and  B  on  some  scale. 

It  may  have  become  apparent  that  the  word  validation  is  being  used 
in  the  EMETF  in  two  somewhat  different  contexts.  One  may  be  called 
validation  for  development.  This  consists  of  whatever  tests  or  comparisons 
may  be  useful  in  checking  out  the  development  of  a  module,  particularly 
the  propagation,  equipment  and  scoring  modules,  as  implied  in  figure  3. 

The  IPM  is  designed  to  be  a  theoretical  model  rather  than  an  empirical 
one.  That  is,  it  performs  the  calculations  textbooks  and  .Research  papers 
give  in  explanation  of  what  happens.  It  is  not  supposed  to. store  empirical 
data  on  what  has  been  observed  at  ■<  arious  times  and  places,  and  regurgitate 
the  solution  to  the  stored  problem  v/hich  is  most  similar  to  the  desired 
problem.  While  this  theoretical  approach  is  the  cause  of  considerable  grief 
during  development,  the  advantages  of  a  good  theoretical  model  over  an 
empirical  one  are  evident. 

However,  as  it  happened,  we  could  not  wait  the  millenium  without  use¬ 
ful  outputs  from  the  facility.  Our  sponsors  presently  began  to  clamor  for 
results,  and  results  we  had  to  produce,  even  though  we  knew  that  none  of 
the  modules  performed  to  our  standards.  We  were  thus,  in  part,  forced 
into  interim  empirical  solutions.  From  this  there  arose  another  concept, 
validation  for  utilization,  a  statement  about  the  goodness  of  our  results. 

We  shall  henceforth  be  concerned  here  with  this  latter  sort  of  validation. 
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A  block  diagram  of  the  entire  test  problem,  figure  4,  will  be  a  helpful 
introduction  to  the  next  section.  This  shows  the  test  transmitter  with  its 
special  test  signal  and  the  propagation  path  to  the  test  receiver  and  scoring 
device  described  later,  labelled  VIAS.  The  figure  also  shows  potential 
*  interfering  transmitters  (IG),  each  of  which  is  supplied  with  normal  modu¬ 

lation,  ile.  ,  voice,  radioteletype,  etc. 

The  two  scales  below  the  diagram  are  merely  subjective  guesses,  and 
no  precise  quantitative  interpretations  should  be  given  to  them.  For 
example,  on  the  adequacy  of  representation  scale,  which  applies  to  the 
IPM,  we  know  that  both  the  test  signal  and  the  test  transmitter  are  repre¬ 
sented  more  adequately  than  is  the  test  receiver.  This  is  because  most 
of  the  various  things  which  happen  to  a  signal  as  it  passes  through  the 
equipment  occur  in  the  receiver.  We  also  note  that  the  receiver  in  turn 
is  more  adequately  represented  than  is  propagation  loss. 

The  bottom  scale  includes  not  only  factors  related  to  less  than  perfect 
representation  in  the  model,  but  also  includes  variation  in  electrical 
characteristics  among  equipments,  time -dependent  variation  over  a  propa¬ 
gation  path,  and  the  like. 

However  inaccurate  these  judgements  may  be,  they  did  provide  some 
guidance  for  separating  the  total  problem  into  parts.  One  easy  choice  to 
make,  and  one  which  is  also  required  by  the  operational  scheme,  consists 
of  fragmenting  the  problem  into  the  interfence  versus  the  non-interference 
cases;  that  is,  study  the  problem  with  and  without  the  interfering  trans¬ 
mitters  activated.  The  balance  of  this  paper  will  be  concerned  only  with 
the  non-interference  case. 

Another  division  point  was  taken  at  the  input  to  the  receiver.  This 
was  selected  on  a  recent  test  for  several  reasons.  One  is,  it  appears 
from  the  diagram  that  the  first  portion  of  the  chain,  from  test  signal  to 
receiver  input,  would  be  basically  a  measure  of  the  ability  of  the  model  to 
predict  propagation  loss.  Another  reason  is  that  better  measurements 
can  be  made  on  the  low  level  signals  at  the  receiver  than  on  the  high  level 
signals  at  the  transmitter.  A  third  is  that  in  the  workshop  we  studied  in 
detail  the  receiver- VIAS  subsystem,  and  here  the  input  to  the  subsystem 
was  of  necessity  the  input  to  the  receiver.  Thus  the  non-interference 
case  was  divided  into  two  segments. 
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From  the  field  facility,  data  on  the  received  signal  can  be  obtained 
from  a  number  of  transmitters  at  one  or  more  receiver  sites,  each  trans¬ 
mitter-receiver  combination  defining  a  path.  In  the  IPM,  these  paths  may 
be  simulated  and  the  computer  signal  at  the  receiver  obtained  for  each. 

Thus  we  generate  a  set  of  bivariate  data  as  shown  in  figure  5. 

» 

These  data  were  treated  by  the  method  of  simple  linear  regression. 
Since  the  regression  will  be  used  to  provide  an  interval  estimate  for  the 
expected  value  of  a  hypothetical  "field"  signal  given  a  model  signal,  the 
latter  was  used  as  the  independent  variate.  The  confidence  band  shown 
is  that  for  the  line  as  a  whole.  In  other  words,  it  is  based  on  the  tabular 
factor 

*2^-  «  rather  than  on  t  _  which  is  valid  for  only  one 

2,  n-2  n-2 ,  1 


prediction  of  the  expected  value  for  Y,  given  X.  This  confidence  belt 
has  roughly  25  percent  greater  width  than  the  one  of  the  same  confidence 
level  which  is  computed  using  the  Student  t.  We  used  this  method  for 
showing  how  well,  on  the  average,  the  IPM  was  predicting  path  loss. 

The  first  specific  question  directed  to  the  Panel  arises  here.  Is  * 

there  a  method  for  providing  an  interval  estimate  for  any  number  of 
individual  predictions  ? 

To  lead  into  the  second  question,  further  details  are  helpful.  The 
regression  is  based  on  the  received  signal  measured  in  negative  dbm, 
that  is,  in  decibels  below  one  milliwatt.  This  is  a  measure  of  the  signal 
power  induced  across  the  input  impedance  of  the  receiver.  The  gain  of  * 

the  receiver  antenna  is  thus  included  in  the  signal  power  measurement. 

In  practice,  however,  it  may  be  necessary  to  measure  the  field  intensity, 

a  voltage  impinging  upon  the  receiver  antenna.  Thus  the  dbm  measure 

shown  may  contain  a  computed  element.  Most  likely  this  would  be  com-  « 

puted  once  for  each  type  of  antenna- receiver  combination,  and  would  not 

include  interantenna  variability  or  variable  ground  plane  effects.  Thus 

the  dependent  variate  may  among  other  things  contain  a  fixed,  computed 

component  rather  than  a  measured,  variable  one.  Clearly,  we  need  in 

such  cases  to  assess  the  effects  on  our  predictions. 
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This  ui.her  examples  not  cited  pertain  to  the  general  question  of 
whether  we  do  in  fact  satisfy  the  several  assumptions  inherent  in  the 
regression  model,  which  now  suggests  the  second  question,  If  conven¬ 
tional  regression,  upon  further  examination,  is  not  applicable,  can  the 
Panel  suggest  alternative  approaches?  Remember  that  for  subject 
matter  reasons  we  desire  to  obtain,  at  approximately  this  point  in  the 
chain  of  events,  a  measure  of  how  well  the  IPM  is  performing  its  job. 

I  will  close  this  section  by  pointing  out  that  we  fully  realize  the 
measure  of  our  ability  to  predict  path  loss  over  one  terrain  type,  in  an 
area  of  sparse  vegetation,  is  not  necessarily  indicative  of  how  well  the 
model  will  perform  under  other  terrain-vegetation  combinations,  The 
Army  and  others  are  presently  engaged  in  collecting  propagation  data  in 
vax.jui  areas  of  the  world,  At  present,  however,  the  Arizona  data  are 
all  we  have  to  work  with.  It  is  our  hope  that  by  the  time  suitable  data 
from  other  areas  are  available,  we  will  have  established  the  techniques 
to  use  these, 

Before  proceeding  to  .the-  securifiTJbTtTffn  "of  Ihe  non-interference 
chain,  it  will  be  helpful  to  deecribe  the  scoring  device.  The  Voice  Inter¬ 
ference  Analysie  Set  (VIAS),  is  a  commercial  device  designed  to  convert 
signal-to-noiee  type  information  from  the  terminal  end  of  the  receiver 
audio  section  into  a  meaeure  which  is  monotonically  related  to  intelligi¬ 
bility.  The  result  is  the  Articulation  Index  (AI),  The  conversion  is 
accomplished  by  subdividing  the  audio  frequenciee  from  200  to  6100  cycle* 
per  second  into  14  bands,  each  of  which  is  supposed  to  contribute  equally 
to  speech  intelligibility.  In  each  band  the  signal -to-noise  ratio  ie  measured 
during  17  seconds  of  the  30-second  test  period,  For  signal-to-noise  ratios 
of  +18  db  or  higher,  the  aignal-to-noiee  ratio  is  converted  to  unity;  for 
ratios  of  -12  db  or  lower,  the  conversion  results  in  zero.  In  between  +18 
and  -12  db,  the  conversion  is  approximately  a  linear  function  of  the  signal- 
to-noiso  ratio,  The  final  articulation  index  is  simply  the  mean  of  the  14 
increments,  There  are  some  additional  manipulations  involved,  and  a 
special  test  signal  is  required,  but  these  details  need  not  concern  us  here. 
This  device  is  based  upon  studies  by  French  and  Steinberg,  and  by  Beranek. 

It  should  also  be  noted  that  if  a  voice  communication  system  does  not 
possess  the  full  bandpass  of  200  to  6100  cycles  per  second,  the  VIAS  bounds 
the  Al  between  zero  and  some  value  less  than  unity. 
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The  second  segment  of  the  chain  concerns  me  radio  receiver  and  the 
scoring  device.  In  the  ehop,  a  rather  precise  curve  can  be  established 
which  relates  the  signal  power,  developed  across  the  receiver  input 
impedance,  to  the  A1  output,  This  curve  generally  resembles  that  shown 
in  figure  6.  Three  things  should  be  noted.  First,  the  figure  shows 
hypothetical  data  and,  if  anything,  the  point  scatter  is  excessive,  An 
individual  receiver  produces  data  points  which  scarcely  deviate  from  a 
smooth  curve.  Second,  what  information  we  have  indicates  that  variation 
among  receivers  results  primarily  in  a  horizontal  translation  of  the  curve 
by  no  more  than  a  very  few  db.  This  is  apparently  the  result  of  variation 
In  receiver  sensitivity.  Third,  measurements  which  fall  in  the  lowest 
fourth  or  fifth  of  the  A1  scale  are  difficult  to  make,  and  these  exhibit  a 
higher  variability  than  those  resulting  from  stronger  signals. 

It  should  also  be  stated  that  the  effects  of  varying  the  modulation  level 
at  the  transmitter  are  at  present  unknown  in  detail,  but  presumably  are 
very  important. 

The  mathematical  nature  of  the  Al/signal  relationship  is  not  known 
from  theory,  as  far  as  we  have  been  able  to  ascertain.  An  understanding 
of  the  manner  in  which  the  VIAS  operates  on  the  signal  clearly  explains 
the  rounded  corners.  It  also  allows  for  a  strictly  linear  portion  in  the 
descending  leg  of  the  curve,  at  least  for  some  values  of  signal  and  noise. 
Finally,  the  bounds  on  the  functior  are  easily  understood.  Perhaps  this 
is  enough. 

In  practice,  the  probit  transformation  was  applied  to  the  AI  axis  and 
a  reasonable  linear  trend  was  established.  Although  the  potential  applica¬ 
bility  of  the  probit  transformation  is  not  immediately  obvious,  a  study 
by  our  contractor  indicated  that  it  could  be  used.  Confidence  intervals 
were  established  by  the  methods  appropriate  to  problt  analysts,  and  then 
mapped  through  the  inverse  of  the  transformation  to  provide  the  confidence 
belt  shown  in  figure  7, 

The  three  curves  to  the  right  represent  the  fitted  line  and  its  confi¬ 
dence  bands  for  a  specific  equipment  type.  The  single  line  to  the  left 
shows  only  the  fitted  curve  for  another  equipment  type, 

Note  that  the  line  on  the  left  drops  from  maximum  AI  to  zero  over 
a  spread  of  about  25  db,  whereas  the  other  curve  takes  a  little  over  50  db 
to  drop  to  zero,  When  it  is  considered  that  the  power  output  of  the  trans¬ 
mitters  normally  associated  with  these  receiver  types  is  in  the  vicinity 
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of  +40  to  +45  dbm,  we  see  that  a  receiver  over  most  of  the  possible  range 
of  signals  is  either  performing  at  its  best,  or  else  is  not  extracting  any 
intelligence  whatever  from  the  desired  signal.  . 

The  use  of  the  probit  transformation  was  an  expedient.  It  is  clear 
that  in  some  cases  it  is  not  appropriate,  clear  because  the  best  fitting 
probit  line  obviously  does  not  fit  well.  In  particular,  if  there  is  a  consid¬ 
erable  segment  of  the  curve  which  is  linear  in  the  descending  region,  the 
probit  transformation  is  not  suited. 

The  third  question  for  the  Panel  is  this;  Please  comment  on  the 
problem  of  providing  both  point  and  interval  estimates  for  the  functional 
relationship  between  input  power  and  output  AI. 

An  earlier  topic,  the  scoring  device,  dealt  with  a  conversion  from 
an  electrical  measure,  the  signal -to-noise  ration,  to  a  psychoacoustical 
measure,  aural  intelligibility.  The  question  naturally  arises;  Is  the  AI 
scale  a  suitable  measure  of  intelligibility? 

Previous  work,  notably  by  Kryter,  had  shown  that  AI  was  not 
linearly  related  to  the  articulation  score  (AS),  where  the  latter  is  defined 
as  the  proportion  of  words  recorded  correctly  by  a  listener.  Kryter 
showed,  further,  that  different  AS/AI  relationships  were  obtained  depend¬ 
ing  upon  the  size  of  the  word  list.  He  and  others  have  shown  or  suggested 
that  such  other  factors  as  the  type  of  noise,  i.  e.  /  white  noise,  voice 
babble,  or  meaningful  single  voice  interference,  also  affect  the  AS/AI 
transformation.  We  have  recently  verified  that  the  electronic  circuitry 
of  the  communication  equipment  also  affects  this  relationship. 

While  there  are  some  theoretical  results  which  predict  the  functional 
relationship  between  AS  and  AI,  our  position  is  that,  at  present,  the 
relationship  must  be  established  empirically.  Naturally,  we  anticipate 
the  day  when  the  appropriate  theory,  to  include  parameter  values,  has 
been  established  and  can  be  used  to  convert,  in  the  IPM,  from  the  last 
electrical-type  measure  to  intelligibility. 

The  articulation  score,  as  we  use  it,  is  defined  as  the  mean  propor¬ 
tion  of  correct  responses  given  by  five  listeners  to  a  transmission 
involving  one  of  several  50-word  phonetically  balanced  lists. 
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The  experimental  procedure  requires  that  the  word  list  be  trans¬ 
mitted  and  recorded  on  magnetic  tape,  The  transmission  also  includes 
the  special  test  signal  required  for  the  A1  measurement,  For  various 
reasons,  we  now  imbed  the  test  words  in  carrier  phrases,  and  this 
procedure  necessitates  a  transmission  time  of  16  minutes,  Each  tape  is 
scored  in  a  special  listening  facility  by  five  operators.  The  A1  signal  is 
scored  separately  by  the  V1AS,  Thus,  one  transmission  produces  one 
AS/AI  datum  point, 

Figure  8  presents  some  recent  AS/AI  data  acquired  from  different 
equipments,  The  actual  points  are  shown  only  for  the  middle  curve.  The 
scatter  of  points  shown  is  roughly  typical  of  each  curve. 

These  curves  were  supplied  by  our  contractor.  The  center  and  lower 
curves  are  based  on  the  Gomperts  curve  while  the  upper  one  ie  hyperbolic, 
The  Gomperts  curve, 


X 

Y  »  abC 


with  4,  taken  as  unity,  has  been  given  some  theoretical  justification  by 
previous  psychoacoustic  studiss.  It  was  fitted  in  linear  form  by  meant 
of  a  log  log  transformation  in  which  Y  is  l/AS  and  X  ie  AI.  The  transforma¬ 
tion  enabled  simple  linear  regression  techniques,  including  confidence  bands, 
to  be  applied,  The  confidence  bands  were  mapped  through  the  inverse  of 
the  transformation  to  provide  an  approximate  confidence  belt  for  the  line  as 
a  whole.. 

The  next  question  to  the  Panel  is  doubtless  now  evident.  Please  com¬ 
ment  on  the  problem  of  converting  AI  to  AS, 

In  summary,  we  began  with  a  complex  total  problem,  restricted  it  to 
voice  communication,  and  further  restricted  it  to  the  non-interference 
case.  The  non-interference  case  has  been  broken  into  three  segments, 
each  treated  ae  a  regression.  The  dependent  variate  for  one  becomee  the 
independent  variate  for  the  next,  with  the  articulation  ccore  as  the  ultimate 
dependent  variate,  At  present,  approximate  and  conservative  confidence 
limits  cam  be  placed  on  the  expected  value  of  the  AS,  We  are  aiming  for 
the  ability  to  place  exact  confidence  limits  on  the  Individual  AS  predictions. 

My  final  question  to  the  Panel  asks  for  discussion  of  this  problem  of 
ultimate  interest. 


♦ 
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Figure  1.  Artist's  concept  of  the  EMETF  field  facility 
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THE  DESIGN  OF  COMPLEX  SENSITIVITY  EXPERIMENTS 


D.  Rothman  and  J.  M.  Zimmerman 
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1.  INTRODUCTION.  There  is  a  growing  tendency  among  the  practi¬ 
tioners  of  the  art  of  experimental  design  to  allocate  more  of  their  efforts 
to  the  macroscopic  aspects  of  test:. planning.  This  often  results  in  greater 
benefit  than  that  obtained  from  intensive  improvement  of  isolated  experi¬ 
mental  segments.  Very  little  work  of  this  kind  has  been  carried  out  for 
sensitivity  experiments,  however,  despite  the  long  history  of  statistical 
effort  in  this  field,  probably  for  two  reasons.  First,  most  of  the  major 
laboratories  conducting  sensitivity  experiments  have  established  over  the 
years  their  own  traditional  set  of  test  procedures  which  are  relatively 
insensitive  to  variations  in  experimental  objectives.  Secondly,  the 
majority  of  sensitivity  experiments  have  been  somewhat  restricted  in 
scope,  being  limited  to  such  purposes  as  material  screening  or  compari¬ 
son  of  properties  with  those  of  a  standard,  and  have  not  usually  required 
extensive  experimental  planning  and  expenditures. 

Recently  it  has  become  clear  to  many  practitioners  that  there  are 
several  newer  methods  for  the  design  and  analysis  of  sensitivity  experi¬ 
ments  which  deserve  more  substantial  attention,  partly  because  of  their 
intrinsic  merit  and  partly  due  to  the  increased  complexity  and  cost  of 
some  current  programs.  It  was  in  connection  with  one  such  program 
that  the  methods  described  in  this  paper  were  developed,  although  a  sub¬ 
stantial  portion  of  the  material  had  been  previously  formulated  under  a 
NASA,  MSFC  research  contract,  NAS  8-11061,  monitored  by  Dr.  John  B. 
Gayle, 

c 

2.  FORMULATION  OF  THE  PROBLEM.  Consider  a  sensitivity 
experiment  in  which  there  are  n  stimulus  variables,  x^,  x^,  ....  xn> 

and  for  which  the  cost  for  each  test  is  at  least  approximately  known  as  a 
function  of  any  combination  of  these  variables.  For  simplicity,  we  assume 
that  this  cost  is  no  different  if  the  test  response  is  positive  (1)  or  null 
(0).  Given  a,  suppose  that  the  goal  of  the  experiment  is  to  estimate  a 
specified  portion  of  that  n-1  dimensional  surface  on  which  the 

probability  of  a  positive  response,  M(x^,  .  .  .  ,  x^),  equals  a.  Our 

analysis  will  be  based  on  a  loss  function,  L,  which  is  made  up  concep¬ 
tually  of  two  terms;  the  cost  of  tolerating  a  specified  variance  in  the 
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estimate  of  S  ,  and  the  cost  of  testing.  The  overall  problem  is  then  to 
a 

find  that  experimental  design  which  minimizes  JL,  the  value  of  L  averaged 
over  those  portions  of  which  are  of  interest. 

The  treatment  of  the  problem  in  this  general  form  requires  a  care¬ 
fully  worked  out  technique  for  the  design  and  analysis  of  multivariate 
sensitivity  experiments  which  is  readily  amenable  to  the  introduction  of 
cost  considerations.  Although  some  algorithms  for  the  design  of 
multivariate  sensitivity  experiments  have  recently  been  developed 
(references  1  and  2),  they  are  extremely  complex  and  do  not  lend  them¬ 
selves  to  the  implementation  of  loss  minimization.  Therefore,  a  simpli¬ 
fication  in  the  structure  of  the  problem  is  required. 


Towards  this  end,  we  replace  the  original  multiple  stimulus -variable 
problem  by  a  hybrid  regression-sensitivity  problem  in  the  following  way. 
We  select  n-1  of  the  stimulus  variables  and  consider  them  as  independent 
variables  in  a  regression  model.  The  remaining  variable  (say  the  n**1)  is 
considered  as  a  stimulus  with  a  possibly  different  response  function  at  each 
combination  of  the  n-1  regression  variables.  Effectively  what  we  are 
doing  here  is  replacing  the  n-variate  response  function  M(x^,  .  .  .  ,  x^)  by 

a  univariate  function  M(x  ;  x. ,  .  .  .  ,  x  )  with  parameters  x. ,  .  .  .  ,  x  . . 


Our  program  will  be  to  estimate,  at  a  set  of  specified  values  of  these 

parameters,  that  value  xa  of  x  for  which  M(x  ;  x. ,  .  .  .  ,  x  )  =  a; 

n  n  n  1  n-1 

each  point  (x, ,  x_,  .  .  .  ,  x  ,  xa)  is  in  fact  on  S  .  Then  we  shall 
1  L  n-1  n  a 

describe  the  effect  of  the  parameters  x^,  .  .  .  ,  x^  ^  by  means  of  an 

a 

ordinary  regression  of  these  variables  on  the  estimates  of  x^  . 


In  a  particular  problem,  the  selection  rof  the  single  stimulus  variable 
from  the  original  set  is  usually  obvious,  being  dictated  by  the  nature  of 
the  experimental  apparatus,  preparation  of  the  test  specimens,  and  long 
standing  practice  (e.  g.  ,  in  drop  tests  involving  several  environmental 
variables,  such  as,  temperature,  orientation  of  the  specimen,  etc.  ,  the 
height  of  the  impactor  would  invariably  be  the  single  stimulus  chosen). 

In  the  present  case,  another  important  consideration  which  may  affect 
the  choice  of  the  stimulus  variable  is  the  relative  influence  it  has  on  the 
cost  of  testing.  Our  optimization  procedure  will  be  based  only  on  the 
regression  variables;  that  is,  we  determine  the  best  combinations  of 
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the  variables  x. ,  .  .  .  ,  x  .  at  which  to  test  in  order  to  minimize  the  loss 
l  n -i 

function  of  the  entire  experiment,  This  macroscopic  type  of  optimization 
does  not  itself  dictate  the  local  or  microscopic  design  for  the  stimulus 
variable  (x^)  at  each  of  the  regression  parameter  level  combinations. 

Thus  it  is  important  in  applying  this  method  to  select  as  the  stimulus  vari¬ 
able  one  which  affects  the  cost  of  testing  as  little  as  possible. 

It  should  be  pointed  out  that  this  general  approach  to  reducing  the 
complexity  of  the  problem  is  not  new,  For  example,  in  1961  Grant  and 
Van  Dolah  described  a  procedure  for  handling  multidimensional  problems 
by  the  use  of  factorial  designs  combined  with  the  simple  up  and  down  method 
(reference  3).  In  our  work,  however,  the  aspect  of  cost  minimitation  has 
been  added,  and  in  addition  a  quantitative  method  for  describing  the  effi¬ 
ciency  of  seneitlvity  experiments  is  developed.  We  treat  these  two  topics 
in  ths  following  ssetions. 

3.  MACROSCOPIC  COST  OPTIMIZATION,  The  regression  model 
relating  the  n-1  variables  x^,  ,  ,  ,  ,  x^  ^  with  the  eatlmatee  of  will 

be  written  in  the  form 

(1)  x“  =  Pq(x)  +  P^x)  +  m  -  +  Pf(x)  +  «  , 

where  x  is  the  v  ctor  (x.,  .  .  .  ,  x  ,),  P.(x)  is  a  mm  of  terms  of  the 
th  ^  n-i  j 

j  degree  in  the  component!  of  x  with  unknown  coefficients,  and  <  is 

a  normally  distributed  random  variable  with  mean  zero  and  (unknown) 

variance  .  Let  N  be  the  number  of  (not  necessarily  distinct)  values 

of  x  at  which  test  sequences  on  the  stimulus  variable  x  are  to  be  run, 

~  n 

The  covariance  matrix,  Q,  of  the  estimates  of  the  coefficients  in  (1) 

can  be  written  in  the  form 

(2)  Q  =  it2  R(x)/N 


where  R  is  a  matrix,  independent  of  7  and  N,  whose  elements  involve 
averages  of  the  components  of  x  over  the  design.  In  treating  particular 
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problems,  one  determines  the  optimum  proportions  of  tests  to  be  con¬ 
duced  at  each  oi  a  certain  fixed  number  ot  optimum  treatment  combina¬ 
tions,  with  N  specifying  the  number  by  which  these  proportions  are 
multiplied  to  obtain  an  actual  design, 

We  have  assumed  that  the  average  cost  per  test  depends  only  onthe 

vector  x  and  not  on  x  or  N  (it  would  depend  on  N  if,  for  example, 
“  n 

there  were  a  setup  cost),  Let  this  cost  be  denoted  by,  C(x),  For  the 
moment  we  suppose  that  it  is  desired  to  obtain  estimates  of  the  function 

xa  (x, ,  ,  .  ,  ,  x  ,)  over  an  a  priori  SDecified  region  U  with  weighting 
n  1  n-1 

function  W(u),  Our  loss  function  is  a  linear  combination  of  the  weighted 
average  of  the  prediction  variance  over  this  region  and  the  cost  of 
testing.  Thus  the  average  loss  is 

(3)  L  «  AN*1*  2  f  V'(u)R(x)V(u)W(u)du  +  BQ(x)  •  ;  N 

Ju  ' 

where  u  ■  (u, ,  u  .),  A  and  B  are  appropriately  chosen  ebn-' 

—  1  n-l 

stante,  and  V  is  a  column  vector  whose  components  are  the  linearly 
Independent  functione  of  the  components  of  x  contained  in  the  quantities 
P  (x),  J  ■  0,  1,  .  ,  .  ,  r  of  equation  (1).  For  example,  in  the  very  eimple 

J 

case  when  x  is  the  scalar  u  and  r  ■  2,  we  have 


I1  ^ 

a  l  u 


In  this  situation  we  have  explicitly 


Q  =  r 


Zx  Z  x 


j  Zx  Zx  Zx 
\  Zx2  Zx3  Zx4 
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R  = 


where 


x 


N 

X 

i=l 


x,( 


etc.  In  the  case  when  N  (or  the  total  coit)  i* 


not  epectfied  in  advance  we  muet  find  that  value  of  N  which  minimise* 
(3).  Since  R(x)  i*  independent  of  N,  on  differentiating  thi*  expression 
one  obtains 


(6) 


N 


opt 


(u)R(x)y(u)W(4)dii/BC(x) 


and  the  associated  value  of  L  is 


(7)  L  -  a r  J  AbJ  V'(u)R(x)V(u)W(u)du  •  C(x) 

u 


Thus,  independently  of  the  values  of  c r,  A,  and  B,  it  le  sufficient  to 
minimise 


(8) 


X?/4or  ZAB 


c(x)  y  V '(u)R(x) V(u) W(u) du 


U 


where  the  right  hand  member  of  (8)  is  proportional  to  the  cost  times 
the  average  prediction  variance  or  "cost  per  unit  of  information".  Not* 
that  this  latter  type  of  loss  minimisation  may  be  accomplished  independ* 
ently  of  N  and  of  the  cost  per  unit  variance  ratio  B/A.  The  value  of 

<rZA/B  is  explicitly  required  only  if  it  is  desired  to  determine  N  ^ 

from  (6).  If  the  total  maximum  expenditure  of  the  test  program  is 
fixed  in  advance,  aa  is  often  the  case,  than  N  is  fixed  and  the  values 
of  a.  A,  and  5  do  not  affect  the  minimisation  of  the  right  member  of  (8). 
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When  the  region  U  over  which  the  prediction  variance  is  averaged  is 
not  specified  a  priori,  the  practical  solution  of  the  problem  becomes  more 
difficult.  For  example,  it  may  be  of  interest  in  some  problems  to  mini* 
mice  the  loss  under  the  circumstances  when  an  estimate  of  the  value  of 

xa(x. ,  ,  .  ,  ,  x  ,)  is  to  be  made  by  extrapolation  to  a  specified  value  of 
n  1  n-l 

xa,  rather  than  to  a  given  value  of  x  =  (x, ,  .  . .  ,  x  In  such  cases, 
n  —  l  n*i 

it  is  generally  not  possible  to  formulate  the  loss  function  explicitly  in  as 

simple  a  form  as  we  have  done  since  the  coefficients  in  the  model  (1)  are 

not  known  in  advance.  In  this  situation  one  may  guess  at  the  values  of  x 

at  which  the  extrapolation  is  to  be  made  and  perform  the  optimisation  for 

a  few  such  poesibilities ,  or,  alternatively,  a  formal  Bayesian  viewpoint 

can  be  taken,  an  a  priori  distribution  of  the  extrapolation  point  made, 

and  the  optimisation' carried  out  formerly  in  terms  of  this  distribution. 

We  will  not  pursue  this  more  difficult  version  of  the  problem  hare, 

although  it  occurs  not  infrsquently  in  practics. 

When  r  ■  1,  and  the  form  of  the  regreseion  and  coat  model*  are 
•imple,  it  is  poaaibl*  to  carry  out  the  minimisation  of  (3)  in  closed  form. 
However,  the  explicit  optimum  value*  of  x  are  not  always  dstermined 
bythis  procedure.  For  example,  ws  have  ehown  (see  reference  4)  that 
when  r  »  1,  there  are  p  regreseion  variables,  and  the  cost  function 
C  is  quadratic,  then  all  that  ii  specified  by  the  minimisation  of  (8)  are 
the  means  and  covariance  matrix  of  the  design  variables.  That  Is,  the 
minimisation  of  (8)  providss 

p  iSlil  +  p  .  .  i* 

**  2  r  2 

constraints  which  ths  optimum  dssign  must  satisfy.  Now  generally 
k(p+l)-l  constraints  are  required  to  define  uniquely  a  design  consisting 
of  k  distinct  points.  When  r  ■  1,  p  ■  1,  for  quadratic  cost,  we  obtain 

-1  ■  2  constraint*;  this  is  one  short  of  the  2(2) >1  ■  3  required  for 

H 

a  unique  two-point  design,  When  r  ■  1,  p  ■  2,  w*  obtain  5  constraint!; 
since  three  points  are  required  to  fit  this  model,  thus  requiring  3(3) -1  ■  8 
constraints,  we  get  a  family  of  optimum  three -point  designs  with  three 
degrees  of  freedom. 

■"Alternatively  we  may  say  that,  for  quadratic  cost,  the  number  of  distinct 
elements  in  the  cross  product  matrix  for  the  design,  less  one,  gives  the 
number  of  constraints  obtained  from  minimisation  of  (8). 
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When  the  cost  function  is  made  up  of  functions  of  components  of  x 
which  do  not  already  appear  in  the  cross  product  matrix  then  one  obtains 
an  additional  constraint  from  the  minimisation  of  (8),  In  fact,  in  the 
general  case  of  any  r  and  p  we  have  made  the  following  conjecture. 

Conjecture:  Let  m  be  the  number  of  distinct  elements  in  the  cross 
product  matrix,  P,  corresponding  to  the  polynomial  model  (1),  of  degree 
r,  Suppose  the  cost  function  C  contains  functions  of  the  components  of 
the  p(=  n>l)  dimensional  vector  x  which  do  not  appear  in  P  (we  refer 
to  this  as  oondition  I).  Then  minimisation  of  the  loss  function  (8)  yields 
m  constraints  for  the  determination  of  the  optimum  design.  If  the  cost 
only  contains  functions  already  appearing  in  P  (condition  II)  then  mini¬ 
mization  of  (8)  provide  a  m-1  constraints, 

Since  a  design  of  k  distinct  "points"  or  treatment  combinations 
requires  k(p+l)-l  constraints  for  unique  determination  we  have  immediately 
the  following: 

Corollary;  Minimisation  of  (8)  results  In  an  optimum  design  consist¬ 
ing  of 

f  i  r 

v*'  1  —  I 

max  j^q,  ]  [  I  points  when  condition  II  prevails, 

s  ,  ,  . 

where  q  ■  E  J  ,  sq  ■  min  [r,p]  ;  q  »  number  of  unknown 

parameters  in  the  model,  and  ]  y  [  denotes  the  smallest  integer  larger 
than  or  equal  to  y.  The  design  is  unique  when  the  quantity  in  brackets  is 
an  integer.  A  formal  proof  of  this  conjecture  may  require  solving  the 
general  minimisation  problem  for  (8),  a  very  formidable  taek.  Even  the 
case  r  ■  1  poses  eerious  difficulties  (see  reference*  4  and  5),  Apart 
from  our  verification  of  the  conjecture  in  the  linear  case  when  condition 
II  prevail*  (reference  5),  we  have  recently  aolved  a  particular  problem 
(using  a  computer  search  procedure)  when  U  is  a  single  point,  r  *  2, 
p  ■  1,  and  the  coet  function  is  exponential  for  all  stimulus  levels  above 
a  specified  value.  A  unique  three-point  optimum  deaign  was  found. 

Applying  the  conjecture  and  corollary  (with  condition  I),  we  find  in  this 
case  that  the  cross  product  matrix  contains  five  distinct  elements  so  that 
indeed  five  constraints  are  obtained  determining  a  unique  three -point, 
optimum  deeign. 


points  when  condition  I  prevails  and 
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In  implementing  this  conjecture  it  is  convenient  to  have  the  explicit 
relation  between  m,  r  and  p,  For  example,  for  small  vain#*  of  the 
l»i.ier  we  have  the  following  table, 


m 


1 

2 

3 

1 

2 

4 

6 

2 

5 

14 

27 

3 

9 

34 

83 

Value  s  of 

m-1 

,  J 

2  r  \ 

1  p  ] 

i  .  J  ■ 

>o 

J  ) 

U  j 

min(2r  ,p) 


Thus,  for  example,  if  the  regression  model  is  cubic  in  three  variables 
and  condition  II  prevails,  one  would  expect  to  find  a  unique  21-polnt  optimum 
design,  Note  that  in  thia  case  q  ■  20,  ao  that  the  number  of  required 
points  ia  greater  than  the  number  of  unknown  parameter!, 


Despite  the  formidable  nature  of  an  explicit  closed  form  minimisation 
of  (8)  in  the  general  case,  numerical  minimisation  procedure  may  not 
require  excessive  effort.  For  example,  the  recently  conducted  study 
referred  to  above  (r  ■  2,  p  ■  1)  only  took  a  few  minutes  to  run  on  an  IBM 
7094  computer, 


4,  BLOCKING  QF  THE  TESTS  AND  THE  GROWTH  OF  INFORMATION, 
Suppo e •  we  have  obtained  an  optimum  k-point  design  by  the  method*  outline d 
above.  The  order  in  which  these  groups  of  tests  are  to  be  conducted  is 
usually  dictated  by  epecific  characteristics  of  the  particular  program. 
Generally  the  "least  expensive"  treatment  combination  or  point  (from  the 
point  of  view  of  C(x)  )  will  be  explored  first,  then  the  next,  and  so  on 
until  the  moit  expensive  point  is  arrived  at,  We  will  not  consider  this 
queition  further  here,  but  next  turn  our  attention  to  the  design  of  the  indi¬ 
vidual  group  of  sensitivity  tests  at  sach  of  the,  say,  q  optimum  treatment 
combinations  of  the  regresaion  variables. 


Sensitivity  experiments  are  most  efficient  when  they  are  purely 
sequential,  tines  in  this  situation  one  can  reflect  carefully  on  all  previous 
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results  before  selecting  the  next  test  level  for  the  stimulus  variable,  But 
if  the  experimenter  is  required  for  reasons  of  economy  or  manufacturing 
tirna  limitations  co  order  batches  of  test  materials  with  specified  (not 
necessarily  equal)  stimulus  levels  (as,  for  example,  in  solid  propellant 
critical  diameter  studies),  then  It  is  necessary  to  consider  the  question 
of  "block- sequential "  sensitivity  experiments  and  to  evaluate  the  expected 
loss  of  information  implicit  in  this  mode  of  operation  relative  to  the  usual 
purely  sequential  test  procedure, 

To  discuss  block-sequential  designs  we  will  require  a  characterisation 
of  the  amount  of  information  available  before  the  entire  group  of  teats  is 
conducted  (from  previous  studies,  etc.).  This  prior  information  will  be 
expressed  as  that  number  of  equivalent  asymptotically  optimal  tests  which 
would  provide  the  same  asymptotic  information.  Our  approach  will  be 
based  on  the  use  of  asymptotic  expressions  to  characterise  the  growth  of 
information  in  sensitivity  experiments.  Attention  will  be  limited  to  the 
case  in  which  the  response  function  is  a  normal  edf,  and  to  simplify  the 
calculations  we  will  assume  that  the  sole  aim  of  the  tests  Is  to  estimate 
the  median  critical  stress  level  (i.s.  ,  a  »  50%).  Our  analysis  will  be 
carried  out  without  actually  specifying  the  test  levels  to  be  used  in  each 
of  the  blocks,  although  it  is  known  (see  reference  6)  that  for  this  type  of 
experiment  any  test  sequence  converging  to  the  median  is  asymptotically 
optimal  In  terms  of  efficiency  In  estimating  the  median.  Evaluation  of  the 
validity  of  the  asymptotic  theory  for  small  sample  else  is  currently  being 
studied  by  means  of  simulation. 

Efficiency  and  Growth  of  Information.  Suppose  we  have  a  cumulative 
normal  rtipme  function  with  (unknown)  parameters  p  and  o r  ,  Let  p 
denote  the  maximum-likelihood  estimate  of  p  .  Consider  a  design  with 
T  testa  whose  goal  le  the  estimation  of  p  .  An  asymptotic  expression  for 
the  variance  of  p  (as  T  —  •«)  is  given  by  (reference  7) 

(9)  (T)  ~  C2v  Z/(C0C2-C*) 

where  y^  ■  the  level  of  the  stimulus  variable  on  the  itn  test, 
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(yi-n)A  . 

zi B 

i  -tf/a 

/Zir  8 

(10) 

Pi 3 

J___  r 1  -u2/2 

V  2ir  J  8  dU 

-  OC 

= 

1-Pi  - 

V 

*f/piqi  ' 

V 

X  Vtj  . 
i«l  1 1 

♦ 


Since  the  goal  of  th*  experiment  at  each  of  the  q  optimum  regression 
points  is  the  estimation  of  fi  ,  it  is  not  unreasonable  to  restrict  attention 
to  designs  which  are  asymptotically  symmetric  with  respect  to  |x  ,  Then 

cL~o  , 

<r?(T)^r2/C0  . 

2 

It  has  been  shown  (references  6  and  7)  that  7 »  (T)  is  asymptotically 
minimised  when  t^  ■  0,  i  ■  1,  ....  T;  this  ,  minimum  value  is 


^(T)-  (tr/2)o-  2/T 


The  asymptotic  information  after  T  tests,  1^,  may  be  expressed  by  the 
reciprocal  of  the  variance  of  ^  ,  or 
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(11) 


T 

=  E 
1=1 


-2/  2 
Z.  /n  a. * 

i  ■  i  "i 


Thus  the  information  contribution  of  &  test  at  t  =  t  is  given  by 

-t2  2 

(12)  I(t)~  «  /2irpqff 


Since  this  is  maximised  at  t  »  0,  where  we  have 


(13) 


1(0)  ~  2/ttt2 


the  efficiency,  defined  as  the  relative  information  of  an  individual  test 
at  stimulus  level  t,  may  be  written  as 


(14) 


E(t)  •  I(t)/l(0)~  e_t  / 4pq 


The  function  E(t)  is  tabulated  below  for  selected  values; 


Table  1 


1*1 

0  ,1  .2 

,  5 

,75  1,0 

1.  5 

2.  0 

3.  0 

4.  0 

5.  0 

E(t) 

1  .9964  ,9856 

.  9127 

.81  28  .  6888 

.4226 

,  2060 

.  0229 

,  00089 

.  000012 

It  may  be  noticed  that  the  efficiency  declines  rapidly  in  the  range  .  75  <  |t|  <  2. 
Tests  for  which  |t  |  >  3  are  very  inefficient  in  the  long  run,  although  they 
may  provide  a  large  fractional  increase  in  information  early  in  the  experi- 
msnt. 

In  order  to  derive  an  expected  value  for  E(t),  we  express  it  in  a  more 
explicit  form.  It  can  be  shown  that  the  following  expansions  are  convergent 
for  all  values  of  t; 
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P  =  l/2  +  t/Z^IT  -  t  ^  /  feV"  2  TT  +  t^/4(h/"  Z  TT 

(15)  -  t 7 / 3 3 fe V 2 it  +  t9/ 3456V" 2 u  -  tll/42240/ 2tt  +  tU/599040/ 2tt  -  , 

,  A1*'.  q  =  1/2  -  t/v^ 2-rr  +  t3/6vr 2tt  -  t5/40/2ir 
I 

(1.6)  +  t?/ 336V 2it  -  t9/3456/ 2 it  +  tll/42240/ 2tt  -  t13/599040vr 2tt  +  ,  , 

Therefore 

pq  =  l/4  -  t 2  /  2  tt  +  t”^ / 6 tt  -  7t^/l80rr  +  t^/l40n 

(17)  -  8  3t10/7 5600tt  +  7 3t12/498960tT  -  523t' 4/30270240tt  +  ,  ,  .  , 
and 

E(t)  e  /(l-2t2/TT+2t4/3TT-7t^/45TT+t^/35TT 

(18)  -  83t10/l8900TT+73t12/l24740TT-523t14/7567560TT  +  .  .  ,  )  . 

We  have  finally 
2 

e*  E(t)~  1  +  2t2/ tt  -  2t4/3TT  +  4t4/TT2  +  7t6/45rr  -  8t6/3Ti2 
+  8t6A3-  t8/ 3 5rr  +  16tS/l5rr2  -  8t8/TT3  +  16t#A4 
+  82t1O/l89O0TT  -  304t10/945TT2  +  68t10/l5ir3  -  64t10/3ir4 
+  32t10/Tr5  -  7  3t12/l24740ff  +  1132tl2/l4173ir2  } 

-  356t12/l89TT3  +  704tI2/45ir4  -  I60t12/3r5  +  64t12/rr6 

r  ■ 

+  532tU/7567  560r  -  296t14/l7 325-n2  +  599tl4/94 5ir 3 

-  7808t14/945TT4  +  48tU/TT5  -  128t14/-rT6  +  128tl4/ir7  +  ,  ,  ,  . 


(19) 
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In  general,  since  |u  and  <r  are  unknown,  we  are  uncertain  as  to  just 
which  value  of  t  we  are  testing  at;  let  this  uncertainty  be  represented  by 
a  density  f(t)  with  mean  M  and  variance  U.  Since  we  are  trying  to  teet 
a.  x  =  (i  (t  =  0),  and  since  ^  is  unbiased  and  asymptotically  normal, 
we  have 

(20)  M  =  0 

(21)  U£  <r{2  =  <t2(T)A  2  . 

Than  the  expected  teet  efficiency  ie  given  b> 

m 

Wf  5  f  E(t)f(t)dt 


using  the  substitution  v  =  U/(2U+1).  Because  of  (19)  this  integral  can  be 
thought  of  as  a  sum  (with  coefficients  given  by  (19))  of  the  even  central 
momenta,  M,_,  of  a  normal  distribution  with  variance  v.  We  have 

MQ  =  1  , 

M2n  *  (2n-l)vM2n,2  ■  . 


(23) 
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2  3  4 

from  which  it  follow*  that  =  v,  *  3v  ,  Mg  =  15v  ,  Mg  =  105v  ^ 

M1q  =  945v5  ,  M12  ■  10395v6,  and  M14  *  135135v7,  Then  from  (22)  and  (19) 

we  have 


2 

—  v 

TT 

'  TT 

2  1 

‘  ”  , 

’83 

304 

4284 

[20tt 

'  2  + 

3 

.  Tf  TT 


[665280  554400  ,162624  19580  .  12452  73  1  ..6 

6 - 5~ +  “  r”  +TTT‘  'IST  v 

'it  TT  TT  TT  15ff 


523  11544  85657  1116544  .6486480  17297280 

+  .iir-r*- 1 - r~+  t  t 


17297280 \  7 


=  [l  +  .  636620v  +  .  579234v2  +  .  560060v3  +  ,  548604v4 

(24)  +.  539890v5  +  ,  53422  3V6  +  .  530582v7] //lu+T.  . 

Thie  function  i*  tabulated  below  for  selected  value*: 

Table  II 

U  I  0.0  I  0.  2  I  0.  sL  oil.  5  Is.  0I3.0I4.  o|6.o[s.  0  |l0,  0  Il5.  0  20,  0  40,  0  100.  0 


63  .  55  .49  .  42  ,37  .34  .28  .25  ,  18  ,11 


Elt7 w  1  -  .  3634U 


=»  I  u- 
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ia  aufficiently  accurate,  For  large  valuee  of  U  we  have  v  =  .  5  and  numer¬ 
ical  evaluation  of  equation  (24)  leade  directly  to 

(26)  EftT*  1.132/Vu  , 

At  thie  point  we  have  only  an  asymptotic  formula  (24)  for  computing  the 
expected  efficiency  of  a  new  teet,  given  tr  £  ,  But  denoting  by  Eq  the  prior 

information  in  termB  of  equivalent  efficient  teate,  and  by  E^  the  efficiency 

of  the  i**1  teat,  we  have  from  our  definition  of  the  information  after  T 
teat*  that 

(27)  IT'~IT-1  +  ' 

Then  one  can  ehow  by  an  elementary  induction  that 

2  T 

us)  iT — r  Ei  ■ 

■ncr  i«0 

2  2  T 

»ft(T)-(ir/2)»  /  *  E  . 

r  i«0 

Equationa  (21)  ,  (24),  and  (29)  can  be  uaed  to  asymptotically  describe  the 
growth  of  information  in  senaltivity  experiments. 

Of  these  equatione,  only  (21)  it  exact,  Equation  (29)  is  aaymptotically 

valid  aa  ET-  E.  goes  to  Infinity,  which  will  happen  if  and  only  if  Efl 
i»0  i 

and /or  T  beeoma  arbitrarily  large.  Equation  (24)  holds  asymptotically 

on  the  J+lBt  test  ae  X  E^  goes  to  Infinity,  in  which  case  U  goes  to 

saro.  But  note  that  (24)  and  (29)  do  not  give  unreasonable  results  even  for 
large  valuee  of  U  and  for  small  values  of  E~  and  T.  Thus  we  ehall 
attempt  to  draw  tentative  conclusions  even  in  the  latter  caeee. 


or 

(29) 
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In  the  following  example  we  see  how  alowly  the  individual  teat  effi¬ 
ciencies  increase  in  the  course  of  a  purely  sequential  sensitivity  experi¬ 
ment.  Note  that  we  never  have  to  specify  the  Individual  test  levels  in  this 
line  of  reasoning, 

2  2 

Example ,  If  E^  ■  .10,  then  from  (29) ,  <r^(0)'w  15.71ff*  ,  From  (21), 
U~15,71,  andfrom(24),  E^  ~  ,275,  Continuing  in  this  manner,  wo  have 
(1)  ~  4-  19«r2, 

U  -  4.19,  E2  ,486, 

U  *-  1,  82,  Ej  ~  .  647, 

U~  1.  042,  E.  ~  .748, 

4 

U-.696,  Ej-,811, 

U  ~  .  512,  _  .  851, 

U  -  .401,  E?  —  ,  878, 

U  -  .  328,  E0  ^  ,  899, 

U-  .276,  3G  _  .  911,  etc. 

The  above  asymptotic  theory  ha*  been  tested  by  means  of  a  computer 

2 

program  for  simulating  sensitivity  experiments.  The  value  of  or^  given 

by  this  theory  is  much  more  realistic  than  the  value  (ir/2)  *r^/X ,  but  still 
sometimes  conservative  by  a  factor  of  three.  , 

Block-Sequential  Designs,  Now  let  us  introduce  the  notion  of  a  "block- 
sequential"  design,  in  which  each  block  of  teats  is  planned  after  all 
previous  test  results  have  been  analysed.  To  "stage"  such  an  experiment 
means  to  assign  sample  sices  to  each  of  a  given  number  of  blocks,  given 
the  total  sample  sice  T.  The  "optimum"  staging  of  a  block- sequential 
sensitivity  experiment  is  that  staging  which  produces  the  greatest  expected 
gain  in  information.  Using  the  asymptotic  methodology  derived  above,  we 
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have  rnmnuted  a  table  of  optimum  strains  for  2 -block  sensitivity  experi¬ 
ments  for  total  sample  sizes  up  to  34  and  for  two  different  amounts  of 
prior  information, 


Table  III 


Sample  Size  of 

First  Block 

2 

3 

4| ' 

II  ■■MM 

6 

’m  In 

A 

Total  Sample 
Sizes  for  which 
the  Given  F ir st 
Block  is 
Optimum 

E0  =  .°2 

2-3 

4-7 

8-11 

12-15 

16-19 

20-24 

25-29 

30-34 

Eg  =  2,  00 

2-3 

4-6 

7-10 

11-14 

15-19 

20-25 

26-31 

32-34 

For  example,  if  13  tests  permitted  were  specified  at  the  particular  combi¬ 
nation  of  regression  variables  under  consideration,  the  optimum  2-block 
design  would  call  for  4  tests  in  the  first  block  and  9  tests  in  the  second, 
over  the  given  range  of  values  of  Eg. 

It  is  fortunate  that  the  above  results  are  relatively  independent  of  Eg, 
because  this  parameter  is  in  practice  very  difficult  to  evaluate,  For 
example,  if  our  prior  density  on  p  is  uniform  in  [A,B]  ,  then 

r js  (0)  -  (B-A)2/12  . 

But  to  compute 

Eg  °  (tt/2)  0-  2/«r  ^  (0)  , 

2 

we  must  know  <r  ,  and  such  Information  is  almost  always  unavailable. 

Results  of  the  type  given  in  Table  III  are  not  completely  rfgorous 
even  for  large  values  of  Eg  and/or  T,  since  we  compute  expected 
information  in  the  second  stage  as  a  function  of  expected  information  in 
the  first  stage,  rather  than  in  terms  of  the  distribution  of  this  informa¬ 
tion.  But  the  results  are  all  plausible  and  of  practical  value  precisely 
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because  they  are  similar  for  ■  .  02  and  E^  =  2.CQ.  In  addition., 

the  optimum  block  else  a  are  obviously  right  for  a  total  sample  aiae  of 
-two,  and  the  fraction  in  the  first  block  decreases  relatively  Smoothly  as 
the  total  sample  sise  increases, 

It  should  be  noted  in  passing  that  the  above  machinery  permits  us  for 
the  first  time  to  characterise  experiments  in  which  the  stress  variable 
has  an  Independent  "setting"  error,  such  as  the  projectile  velocity  in 
projectile  penetration  tests,  Let  this  setting  error  be  normally  distrib' 
uted  with  mean  0  and  variance  a  ^  ,  Then  the  only  change  in  the  above 

formulae  is  in  the  expression  for  U,  which  is  now 


2,  ,  2 

r,)/* 


2  g  -  y " 

Example >  Let  v*  ■  2r  ,  Thsn  the  asymptotic  efficiency  oi  even  a: 
purely  sequential  design  in  this  case  is  only  63%,  since  U~2  (see  Table 
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FACTORS  AFFECTING  SENSITIVITY  TESTING 


James  R,  Kniss,  and  Warren  S.  Wenger 
Ballistic  Research  Laboratories 
Aberdeen  Proving  Ground,  Maryland 


INTRODUCTION ,  Sensitivity  testing  is  frequently  utilised  by  the  army 
in  evaluating  the  sensitivity  and  consequently  the  reliability  of  percussion 
primers,  Since  such  primers  are  used  rather  extensively  in  nuclear 
warheads,  missiles  and  conventional  munition  systems,  their  functioning 
characteristics  frequently  have  an  important  bearing  on  the  reliability 
of  these  systems,  However,  knowle  dge  of  these  characteristics  of  the 
primers  and  consequently  of  their  reliability  under  varying  temperature 
and  impact  conditions  is  often  rather  limited, 

In  a  recent  study  involving  the  reliability  of  a  nuclear  weapon  system, 
the  effect  of  temperature  and  firing  pin  impact  velocity  on  the  reliability 
of  initiation  of  a  primer  became  of  interest.  This  problem  arose  as  a 
rnsult  of  the  procedures  currently  being  used  to  test  the  system  in  which 
a  particular  type  of  primer  is  used,  These  procedures  do  nbt  Include  a 
test  of  the  primer  itself,  but  the  firing  pin  which  initiates  the  primer  is 
tested  to  determine  whether  the  kinetic  energy  produced  equals  or  exceeds 
a  specified  level  of  kinetic  energy,  The  test  results  thus  far  obtained 
indicate  that  the  level  of  kinetic  energy  specified  is  not  compatible  with 
the  sensitivity  of  the  primers.  For,  although  a  considerable  number  of 
firing  pins  have  failed  to  produce  the  specified  level  of  kinetic  energy, 
in  subsequent  tests  none  of  them  has  failed  to  fire  a  primer,  Further* 
more,  the  results  of  the  primer  testing  that  has  been  done  indicate  that 
the  required  kinetic  energy  is  dependent  upon  the  temperature  of  the 
primer.  It  has  also  been  suggested  that  the  sensitivity  of  the  primers 
might  not  be  a  function  of  kinetic  energy  alone,  but  might  also  be  a  func¬ 
tion  of  the  impact  velocity  of  the  firing  pin.  If  this  relationship  does 
exist  then  any  primer  test  fixture  should  be  designed  to  simulate  the 
stroke  velocity  of  the  firing  pin  normally  used  to  detonate  that  type  of 
primer. 

As  a  result  of  the  questions  that  arose  from  this  testing  problem,  a 
test  was  designed  that  would  (1)  measure  the  sensitivity  of  the  primers 
under  standard  conditions,  (2)  determine  the  effect  of  strike  velocity  upon 
the  kinetic  energy  required  to  function  the  primer,  and  (3)  determine  the 
effect  of  temperature  upon  the  kinetic  energy  required  to  function  the 
primer, 
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DISCUSSION  OF  TEST  DESIGN,  The  test  was  to  be  conducted  using 
the  Bruceton  Up-and-Down  method  of  sensitivity  testing.  This  method  has 
been  ueed  for  years,  primarily  in  evaluating  the  quality  or  changes  in 
quality  of  conventional  primers.  However,  no  record  could  be  found  of 
any  test  conducted  where  .ne  strike  velocity  and  temperature  effects  were 
investigated.  Most  past  tests  wers  conducted  at  ambient  temperature  and 
&  single  weight  ball  was  dropped  throughout.  The  height  at  which  50%  of 
the  primers  would  function  was  estimated  along  with  the  variability  ih 
this  height.  The  results  were  used  primarily  to  detect  trends  due  to  age 
and  to  detect  lot-to-lot  variability. 


For  this  test  the  conventional  procedures  were  modified  in  that  four 
different  weight  balls  and  four  differsnt  conditioning  temperatures  wars 
used.  A  conventional  type  primer  (the  MK2A4)  was  used  since  tha  primer 
in  question  was  not  available  and  the  MK2A4  is  of  a  similar  type,  Thi 
primers  ware  tested  according  to  the  following  design: 

Ball  Weight  (on. ) 


If 


8 


12 


If 


16 


70 


‘ill 

[112 


121 

[122 


131 

:132 


141 

‘142 


25 


Temp.  (°F)  -20 


-65 


211 
‘  212 


311 

'312 


X 


411 

412 


441 

‘442 


Where  the  x'e  represent  the  drop  height  at  which  50%  of  the  kth 
■  ample  of  primers  conditioned  at  the  ith  temperature  and  using  the  jth 
ball  weight  will  function,  However,  it  ie  obvious  that  the  drop  height 
will  be  affected  by  ball  weight;  and,  of  course,  we  are  not  interested  in 
thi*  obvious  relationship, 
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In  order  to  obtain  the  relationship  in  which  we  are  interested,  it  will 
n<*r»««»ry  to  convert  three  drop  heights  to  kir.alic  %>i uaing  the 
following  transformation 


'ijk 


Wj*ijk 


where  w  is  the  ball  weight  in  ounces  and  y  is  kinetic  energy  in  inch- 

j  ‘J  * 

ounces.  The  kinetic  energy  obtained  will  be  a  function  of  the  sensitivity 
of  the  primer  and  should  be  unaffected  by  the  strike  velocity  or  temper¬ 
ature  if  theue  factors  are  indeed  insignificant. 


It  is,  therefore,  possible  to  hypothesise  that  if  kinetic  energy  ie  the 
only  factor  affecting  the  sensitivity  of  the  primer  the  analysis  of  the 
data  should  reveal  no  significant  effects  due  to  either  temperature  or 
ball  weight,  Should  ball  weight  affect  the  required  energy,  it  could  be 
further  hypothesised  that  thia  difference  is  due  to  the  impact  velocity  of 
the  firing  pin. 

At  firet  glance  the  above  deeign  would  suggest  that  a  simple  two-way 
classification  of  variable*  should  be  performed,  However,  it  wae  sus¬ 
pected  (and  later  confirmed)  that  the  homoganiety  of  variances  assumption 
which  is  necessary  for  this  type  of  analyeie  m4ght  not  be  met. 


This  lack  of  homogeniety  becomes  Intuitively  obvioue  when  it  is 
considered  that  the  change  in  kinetic  energy  per  unit  change  in  drop 
height  is  greater  for  the  heavy  ball,  This  would  imply  that  the  lighter 
ball*  yield  better  estimates  of  drop  heights  and  consequently  the  vari¬ 
ability  associated  with  such  eetimatea  will  be  imaller  for  lighter  balls. 

If  the  data  were  analysed  a*  it  is,  erroneous  result*  might  be 
obtained  aa  to  the  significance  of  the  main  effects  aa  well  as  of  any 
interaction  that  might  exist  between  ball  weight  and  temperature, 


It  was,  therefore,  planned  to  break  the  results  down  and  firet  work 
with  ball  weight  va,  kinetic  snargy  at  each  level  of  temperature, 


This,  of  course,  doee  not  eolve  the  problem  of  the  lack  of  homo¬ 
geniety,  and  it  ia  necessary  to  correct  for  this  condition  before  progress¬ 
ing  further,  This  may  be  accomplished  by  computing  the  within  cell 
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variation  for  each  ball  weight  and  attempting  to  obtain  the  standard  devia¬ 
tion  as  a  function  of  required  kinetic  energy  over  columns,  (ball  weights) 
i.  e .  : 


if  wu  can  obtain  a  relation  <r  =  £(pt) 
then  J  •  d|a  is  an  appropriate 

transformation  that  will  transform  the  data  so  that  the  variability  will 
be  independent  of  the  ball  weight, 

It  is  now  possible  to  determine  the  relationship  of  ball  weight  to 
kinetic  energy  throu  ghthe  use  of  a  simple  least  square  analysis  per¬ 
formed  on  the'tr&nsformed  data,  However,  it  should  be  understood  that 
the  transformed  data  should  be  used  only  for  purposes  of  significance 
testing  and  that  the  actual  relationships  should  be  represented  by  the 
un-transformed  data. 

It  must  be  determined  whether  this  relationship  differs  for  eacV  or 
any1  of  the  temperatures,  If  it  doss  differ,  is  the  difference  only  in 
intercept,  only  in  slope,  or  in  both  Intercept  and  slops?  If  it  differ!  only 
in  intercept  the  differences  are  constants  over  ball  weight  and  all  the 
data  may  be  corrected  back  to  ambient  temperature  eo  that  a  final  rela¬ 
tionship  may  be  represented  at  ambient  temperature  (or  at  any  other 
tenqfcerature  within  the  range  of  the  test  that  may  be  of  interest).  If 
thsf  relationship  differs  in  slops,  an  intsraction  between  temperature 
and' ball  weight  ia  indicated,  and  the  relationship  will  necessarily  be 
represented  for  each  temperature  or  range  of  temperatures  ovsr  which 
the  slopes  art  homogsnsous. 

In  any  case  a  final  representation  of  required  kinetic  energy  (to 
function  50%  of  the  primers)  will  be  obtained  and  will  be  of  the  form 
E  ■  a  +  bw  +  cw2  +  ,  , ,  (since  a  simple  least  square  fit  is  being  used), 
However,  the  relationship  of  kinetic  energy  (E)  to  strike  velocity  is  of 
primary  interest;  and,  therefore,  the  ball  weight  (w)  in  the  above  equa¬ 
tion  must  be  converted  to  strike  velocity  (v), 


This  may  easily  be  done  since  a  direct  relationship  exists  between 
weight  and  velocity  for  any  free  falling  body  (neglecting  air  resistance, 
etc,  ).  For  example,  aeeuming  a  linear  relationship  between  required 
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energy  and  ball  weight,  the  rel&tlonahip  between  required  energy  and  atrike 
velocity  may  be  obtained  a*  follow*- 

Given  the  relation; 

(1)  E  *  a  +  bw 

and  the  equation*  for  free  falling  bodiea: 

(2)  S»(l/2)gt2 

(3)  V  a  gt 

(4)  E  =  WS 

where  E  ■  Energy  (in.  -ot.  ) 
a,  b  ■  conatanta 

S  ■  Drop  height  (in,  ) 

g  *  acceleration  of  gravity  (384  in.  /eae.  2) 
t  ■  time  ( aec. ) 

V  ■  Strick  Velocity  (in,  /ate,  ) 

Solving  equation  (2)  for  t;  t  *  \/  2S/g  aubatituting  thie  value  for  t  In  (3); 

V  ■  gt  ■  gV" 2S/g  ■  V  2gS 

then  aubatituting  for  S;  S  ■  E/W  give* 

(5)  V  ■  V  2gE/\V 

and  aolving  (5)  for  W;  We  2gE/V2 
finally  aubatituting  in  (1)  and  aolving  for  E 
E  «  a  +  b  (2gE/V2) 
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gives 

a  V2 

(••6)  E  ■  — = -  ,  the  desired  relation  , 

V  -2bg 


The  final  relationship  of  kinetic  energy  to  temperature  may  be 
obtained  in  a  similar  manner.  However,  in  this  case  there  is  no  reason 
to  believe  that  the  variances  will  not  be  homogeneous. 


DISCUSSION  OF  TEST  RESULTS,  In  order  to  conduct  this  test,  32 
samples  o*  MK2A4  primers  were  fired,  each  sample  being  comprised 
of  40  primers.  Using  the  Bruceton  up  and  down  method,  the  32  estimates 
of  50%  points  were  as  follows  for  each  combination  of  temperature  and 
ball  weight  used: 


Temp,  (°F) 


50%  Points  (in  inches) 
Computed  frcm  Up  and  Down  Tests* 
of  Primer,  Percussion,  Mk2A4 

Approximate  Ball  Weight  (ot) 


4 

8 

12 

16 

70° 

8, 7500 
9,4167 

5.  3289 
5,4250 

3,1375 

3,7829 

2,4375 
3.  0329 

25° 

9.  7361 
9.4342 

4.  8289 

5.  2500 

3.  5461 

3.  6591 

2,  6118 
3.1125 

• 

N 

O 

o 

10.0000 

9, 5000 

5.5395  . 
5.  6310 

3.  5417 
3.4934 

2,  7875 
3,1125 

-65° 

9, 8000 
10. 0109 

5.4868 

5.  5500 

3.9500 

3. 6500 

3.1000 

3,4539 

’’A  sample  of  40  primers  was  used  for  each  of  the  32  tests. 


Obviously  thero  is  a  correlation  between  ball  weight  and  the  50% 
points  of  drop  height.  But,  of  course,  this  is  not  very  useful. 

To  get  a  meaningful  basis  for  comparison,  these  heights  were 
converted  to  the  equivalent  values  of  kinetic  energy  by  multiplying  each 
height  by  the  corresponding  ball  weight  (exact),  Therefore,  all  further 
analyses  were  performed  using  the  following  values  of  kinetic  energy: 
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Temp,  (  F)  _2Q0 


Kinetic  Energy  (in,  -oz.  ) 
for  above  50%  Points 


uAirnaie  Bail 

W eignt  (oz; 

4 

8 

12 

16 

o 

o 

34.  5562 
37,1892 

42, 4163 

43. 181 J 

39, 6570 

47. 8146 

39. 7800 
49. 4969 

2  5° 

38, 4506 
37.  2583 

38,  ,4365 

41,  7883 

44. 8216 

46.  2498 

42, 6246 
50. 7960 

-20° 

39, 4928 
37,  5182 

44. 092b 
44.  8210 

44. 7659 
44.1555 

45.4920 

50. 7960 

-65° 

38, 7029 

39, 5358 

43.  6732 

44.  1762 

49.  9267 

46, 1348 

50.  5920 
56. 3676 

Since  we  suspected  that  the  variances  within  ball  weights  might  not 
be  homogeneous,  the  individual  cell  variances  were  calculated  and  tested 
for  homogeneity.  This  test  not  only  confirmed  our  suspicions,  but 
indicated  a  rahter  acute  case  of  non-homogeneity, 

Further  investigation  showed  that  the  relation  between  the  standard 
deviation  and  ball  weight  could  be  satisfactorily  reprssentsd  by  a  func¬ 
tion  of  the  form  o’  *  a  +  bx,  And  the  required  transformation,  to 
correct  for  the  observed  non-homogeneity  was  found  to  be  y  ■  2,  63  in 
(-13.  8  +  ,  38x)  i.  e.  by  substituting  ths  2  above  for  x  in  this  equation  we 
obtained  with  homogeneous  variance's, 

Having  obtained  these  y's,  we  could  then  proceed  to  "determine"  the 
relationship  between  ball  weight  and  kinetic  energy  for  each  of  the  four 
test  temperatures.  A  graphical  representation  of  these  relationships, 
together  with  the  data  from  which  they  were  derived,  is  given  in  Figure  I, 

The  dots,  of  course,  represent  the  dtia  points  and  the  lines  the 
linear  relationship  derived  .rom  these  points  using  least  squares  methods. 
Tests,  using  the  transformed  data  (variances  homogeneous),  showed  the 
slopes  of  these  lines  to  be  significantly  different  from  zero.i,  e,  ,  the  50% 
points  of  kinetic  energy  are  a  function  of  the  ball  weight  uaed. 
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Visual  comparison  (see  Figure  II)  indicated  and  a  test  confirmed  that 
the  slopes  of  these  lines  did  not  differ  significantly,  i.  e.  ,  there  was  no 
reason  to  believe  that  the  difference  in  the  50%  point  resulting  from  a 
given  difference  in  ball  weight  varied  with  temperature.  Or,  to  state  it 
differently,  the  results  of  our  analysis  did  not  contradict  the  hypothesis 
that  the  difference  in  the  50%  point  resulting  from  a  given  difference  in 
ball  weight  is  independent  of  temperature. 

If  we  accept  this  hypothesis,  it  follows  that  a  better  estimate  of  the 
effect  of  ball  weight  on  the  50%  point  of  kinetic  energy  should  be  obtained 
by  "correcting"  the  data  for  temperature  and  then  using  the  resulting  (32) 
points  to  obtain  a  single  relationship.  This  was  done,  and  we  obtained  the 
equation  K.  E.  =  33.  51  +  .  81W  as  our  best  estimate  of  the  relationship 
between  kinetic  energy  and  ball  weight  at  ambient  temperature  (see  Figure 
III). 

We  then  obtained  the  desired  relationship,  between  Strike  Velocity 
and  Kinetic  Energy,  by  inserting  the  values  of  a  and  b  from  the  above 
equation  (a  =  33.  51,  b  =  0.81)  into  equation  (6),’  <  ,  This  relation¬ 

ship  v.Aa  determined  to  be:  E  =  33.  51  V^/(V^  -  620.28).  Figure  IV 
shows  a  graphical  representation  of  this  relationship  and  the  points 
obtained  for  each  of  the  32  samples.  The  velocity  and  kinetic  energy 
values  used  in  plotting  the  points  shown  were  "corrected"  for  temperature. 

The  data  was  also  analyzed  to  determine  the  relationship  between 
the  50%  points  of  kinetic  energy  and  temperature.  As  before,  the  cell 
variances  were  tested  for  homogeneity,  this  time  they  passed,  i.  e.  , 
no  evidence  of  non-homogeneity  was  found. 

We  could,  therefore,  proceed  to  determine  the  relationship  between 
temperature  and  kinetic  energy. 

Again  using  least  squares  methods,  we  obtained  the  relationships 
(see  Figure  V)  between  the  50%  points  of  kinetic  energy  and  tempera¬ 
ture  for  each  ball  weight.  (Comparison  between  points  for  4  oz.  and 
16  oz.  makes  it  apparent  why  the  test  in  the  first  case  indicated  non¬ 
homogeneity  of  variances. ) 

Comparison  of  the  slopes  (Figure  VI)  confirmed  that  they  did  not 
differ  significantly.  Thus  again  a  better  estimate  of  the  effect  of 
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temperature  on  the  50%  points  of  kinetic  energy  should  be  obtained  by 
"correcting"  the  data  for  ball  weight  and  using  the  resulting  (32  )  point* 
to  obtain  a  single  relationship.  This  was  done  (with  data  "corrected"  to 
16  oz. )  and  we  obtained  K.  E.  =  48. 375  -  .  012  t  as  our  best  estimate  of. 
the  relationship  botween  kinetic  energy  and  temperature  (see  Figure  VH). 

To  summarize:  We  found  that,  far  the  Mk2A4  Primer,  both  tempera* 
ture  and  strike  velocity  had  a  significant  effect  on.  the  50%  point  of  kinetic 
energy,  i.e.  ,  the  kinetic  energy  required  to  fire  50%  of  the  primers  is 
a  function  of  the  temperature  of  the  primers  and  the  strike  velocity  of 
the  firing  pin  as  well  as  of  the  sensitivity  of  the  primer. 

While  only  the  Mk2A4  primer  was  tested,  we  would  expect  similar 
results  for  other  percussion  primers.  (If  we  had  reason  to  believe 
otherwise  we  would  not  have  used  the  Mk2A4  for  this  test). 

Therefore,  we  feel,  theee  result*  indicate  the  desirability  of 
considering  the  effect  of  primer  temperature  and  firing  £ln  strike 
velocity  on  the  kinetic  energy  required  by  other  primers  to  assure 
tellable  performance.  Also,  the  desirability  in  testing  primers  of 
simulating  the  strike  velocity  of  the  firing  pin  normally  used  to  detonate 
the  primer  is  indicated. 

Further,  one  might  infer  that  investigation  of  the  effect  of  strike 
velocity  should  be  considered  for  sensitivity  testing  in  general. 


A  COMPARISON  OF  RECONNAISSANCE  TECHNIQUES 
FOR  LIGHT  OBSERVATION  HELICOPTERS  AND  A 
GROUND  SCOUT  PLATOON 

Harrison  N.  Hoppes,  Barry  M,  Kibel,  and  Arthur  R,  Woods 
Research  Analysis  Corporation,  McLean,  Virginia 

INTRODUCTION,  The  Field  Experiments  Division  of  RAC  is  attempt¬ 
ing  to  provide  timely  solutions  to  current  Army  problems  involving  tactics 
and  doctrine.  A  major  portion  of  the  Division's  field  activities  have  dealt 
with  helicopter  operations.  1 1  - iJuJLiJJ  During  July  1963  a  research 
team  from  the  Field  Experiments  Division  conducted  a  two-sided,  fiee- 
piay  field  study  with  the  2nd  Squadron,  4th  Cav,  4th  Armored  Division 
to  evaluate  several  techniques  of  helicopter  reconnaissance.  The  results 
of  that  study  were  presented  at  the  9th  Conference  on  the  Design  of  Experi¬ 
ments  in  the  paper  "An  Analysis  of  Helicopter  Reconnaissance  Techniques. 

In  November  1963,  the  Study's  Project  Advisory  Group  requested 
that  a  winter-phase  investigation  be  carried  out.  The  winter-phase 
venture  measured  the  reconnaissance  effectiveness  of  helicopters  employ¬ 
ing  three  reconnaissance  tactics  and  compared  the  best  of  these  tactics 
with  the  performance  of  a  platoon  of  M114A1  Command  and  Reconnaissance 
Vehicles, 

This  paper  describes  the  experimental  design  employed  in  the  winter- 
phase  investigation,  summarizes  the  results  obtained,  and  presents  a 
brief  statement  of  the  study's  conclusions  and  recommendations, 

EXPERIMENTAL  DESIGN.  The  experimental  design  Is  summarised 
in  Table  1,  As  is  indicated  the  three  helicopter  reconnaissance  techniques 
studied  were:  (1)  "high,  "--flying  at  treetop  level  and  maximum  aircraft 
speed,  (2)  "low  with  pop  up,  "--nap-of-the-earth  flight  with  emphaiie 
placed  on  clearing  an  area  before  entering  by  popping  up  behind  terrain 
masks,  and  (3)  "low  with  dismount,  "--nap-of-the-earth  flight  allowing 
the  helicopter  pilot  to  land  and  dismount  an  observer  with  binoculars, 

Single  OH-13  helicopters,  the  vehicle  currently  used  by  the  light-scout 
lection  of  the  air  cavalary  troop, were  employed  on  all  helicopter 
mis  sione , 
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T  \5LE  1 

Winter-Pha»e  Experimental  Rune 


Ground 

Employment 

High 

Number  of  Run*  For: 
Helicopter  Tactic 

Low/Pop-Up  Low/ 

'Dismount 

Ground  Recon¬ 
naissance 

Platoon 

Total 

Runs 

Stationary 

4 

4 

4 

4 

16 

Moving 

4 

4 

4 

4 

16 

8 

8 

8 

8 

32 

The  ground  reconnaissance  platoon  generally  consisted  of  five  MU4A1 
scout  vehicles.  Usually  the  platoon  leader  divided  the  designated  area  or 
route  into  two  sectors  and  coordinated  the  a  civity  of  the  pairs  of  scouts 
operating  in  each  sector.  In  performing  their  assigned  mission,  scout 
vehicle  commanders  frequently  sent  crew  members  forward  on  foot  in  much 
the  same  manner  as  helicopter  pilots  employed  dismounted  observers, 

Like  the  companion  study  conducted  during  July  1963,  the  winter-phase 
investigation  allowed  scout  elements  complete  freedom  in  determining 
paths  of  reconnaissance  and  time  required  to  complete  the  assigned  mis¬ 
sion.  Helicopter  pilots  were  constrained  only  by  the  reconnaissance 
tactic  they  were  instructed  to  employ;  no  restrictions  whatsoever  were 
placed  on  the  ground  reconnaissance  platoon,  Scenarios  were  designed 
to  be  tactically  realistic  and  still  permit  experimental  control, 

Reconnaissance  missions  were  conducted  against  static  and  fluid 
targets.  Scout  elements  performed  area  reconnaissance  missions  against 
stationary  target  complexes  and  route  reconnaissance  missions  against 
fluid  targets.  On  each  area  reconnaissance  mission  scout  elements 
reconnoitered  against  two  target  complexes,  positioned  to  guard  key  ter¬ 
rain  features  and  likely  avenues  of  approach;  each  target  complex  con¬ 
sisted  of  one  M113  APC  and  one  or  two  M114Al's.  On  route  reconnais¬ 
sance  missions  target  vehicles  generally  consisted  of  two  M113's 
simulating  the  point  of  an  armor  column  and  three  or  four  Mli4Al's  pro¬ 
viding  route  eecurity.  Target  vehicles  were  mounted  with  gun  cameras 
and  event  sequence  recorders.  Vehicle  commanders  were  instructed  to 
engage  all  reconnaiasar.ee  elements  acquired.  Scout  elements,  on  the 
other  hand,  were  told  to  bretk  contact  whenever  an  enemy  vehicle  was 
acquired. 
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Throughout  the  paper  the  term  "stationary  runs"  refer*  to  those  exper 
imental  runs  involving  *tationary  target  complexes  and  "moving  runs"  to 
those  involving  fluid  ground  targets,  Similarly,  the  term  "target  vehicle" 
is  used  to  refer  to  the  ground  vehicle*  against  which  scout  elements 
reconnoitered;  it  is  never  used  to  refer  to  reconnaissance  vehicles  taken 
under  fire. 

RESULTS,  The  winter-phase  experimental  design  discussed  above 
was  successfully  fulfilled  between  20  January  and  6  February  1964.  A 
winter  environment  with  snow  cover,  ground  haze,  and  gray  overcast  was 
present  on  all  days  of  field  activity  except  February  5,  6. 

Data,  obtained  from  event  sequence  recorders  and  gun  cameras 
mounted  on  target  vehicle*  and  from  reconnai* sance  element  *ightings 
reported  to  a  central  control  point,  were  analyzed  using  statistical  tech¬ 
niques.  Major  emphasis  was  placed  on  comparing  (1)  the  performance 
of  helicopter*  v*  ground  scout  teams,  (2)  the  desirability  of  flying  low/ 
dismount  vs  high  vs  low/pop-up,  and  (3)  the  effects  of  reconnoitering 
against  stationary  vs  fluid  target  complexes.  The  basic  statistical  tech¬ 
nique  used  in  making  these  comparisons  was  the  analysis  of  variance; 
other  common  statistical  techniques  employed  were  t  tests  and  chi-square 
tests . 

Analyzing  the  results  of  two-sided,  free-play  experiments  conducted 
in  sector  is  often  quite  difficult,  Frequently  the  outcomes  of  a  given 
situation  differ  widely  and  the  number  of  replications  is  small.  At  times 
experimental  variables  cannot  be  controlled  a*  closely  a*  is  statistically 
desirable  if  troop*  and  equipment  are  to  be  utilized  when  they  are  avail¬ 
able.  As  a  result,  no  attempt  was  made  to  analyze  the  experimental  data 
in  a  rigorous  manner.  The  statistical  analyses  did,  however,  provide 
an  orderly  framework  for  studying  the  large  amount  of  data  generated 
during  the  experiment, 

Multiple  measures  of  effectiveness  were  used  in  analyzing  the  experi¬ 
mental  data.  It  was  felt  that  no  eingle  measure  could  adequately  consider 
all  facets  of  the  reconnaissance  mission.  Among  the  most  important 
measures  were  those  dealing  with  acquisitions ,  firings,  and  length  of 
time  required  for  mission  completion,  These  included:  (1)  the  percent 
of  available  targets  acquired  by  reconnaissance  elements,  (2)  the  percent 
of  ground  targets  acquiring  at  least  one  reconnaissance  element,  (3) 
the  total  number  of  times  reconnaissance  elements  were  detected,  (4)  the 
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number  u  1  limes  rcccr.r.ai ; ; ?.nc »  ^l^menta  and  ground  targets  saw  each  other 
first,  (5)  the  average  length  of  interacquisition  advantages  scored,  (6)  the 
percent  of  the  time  the  reconnaissance  element  was  heard  before  it  was 
seen,  (7)  the  average  lay  time  against  scout  elements  (6)  the  percent  of 
reconnaissance  elements  acquired  that  were  taken  under  fire  by  ground 
target  vehicles,  (9)  the  total  number  of  individual  weapon  firings  at 
reconnaissance  elements  (10)  the  number  of  simulated  rounds  fired  at 
scout  elements,  and  (11)  the  time  required  to  complete  reconnaissance 
missions.  Each  of  these  measures  has  its  merits  and  its  limitations.  By 
considering  a  variety  of  measures  the  relative  ability  of  reconnaissance 
elements  to  acquire  targets,  avoid  destruction,  and  provide  timely  infor¬ 
mation  can  be  estimated. 

Summary  data  concerning  these  measures  are  shown  in  Tables  2-4. 
From  these  data  it  can  be  seen  that: 

1.  Helicopters  acquired  about  60  percent  of  the  available  ground 
targets  regardless  of  the  reconnaissance  technique  employed.  Based 

on  the  percent  of  ground  targets  acquiring  a  helicopter,  the  total  number 
of  times  helicopters  were  detected,,  and  the  net  number  of  acquisition 
advantages  scored  against  helicopters,  the  low/dismount  tactic  was 
superior  to  the  other  two  helicopter  tactic9  examined. 

2.  Based  on  the  number  of  firings  and  number  of  rounds  simulated 
against  helicopters,  pilots  employing  the  low/dismount  tactic  also  out¬ 
performed  those  using  the  high  and  the  low/pop-up  tactics. 

3.  On  the  average  it  required  10  minutes  to  complete  missions  flying 
at  treetop  level  and  maximum  OH-13  speed.  Low/pop-up  missions  lasted 
twice  this  long  and  low/dismount  missions  3^  times  as  long, 

4.  The  acquisition  performance  of  a  single  OH-13  helicopter  was 
quite  similar  to  the  performance  cf  a  platoon  of  five  M114A1  scout  vehicles. 
Both  acquired  about  the  same  percent  of  available  targets  and  both  had 

8  net  acquisition  advantages  scored  against  them.  Only  according  to  one 
acquisition  measure  did  low/dismount  helicopters  and  the  ground  scout 
platoon  differ  widely;  helicopters  employing  the  dismount  tactic  were 
acquired  audibly  before  they  were  seen  23  percent  of  the  time  compared 
with  only  3  percent  of  the  time  for  ground  scouts. 
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5.  On  stationary  runs  tne  acquisition  and  firing  measures  listed  in 
Table  3  indicated  that  the  ground  reconnaissance  platoon  outperformed 
the  single  OH-13  helicopter  flying  the  dismount  tactic.  .  On  moving  runs, 
however,  the  low/dismount  helicopter  tactic  was  more  effective  than  the 
ground  scout  vehicles. 

6.  In  terms  of  all  11  performance  measures  summarieed  in  Table  3, 
reconnaissance  elements  were  more  effective  against  the  fluid  target  com 
plex  than  against  the  stationary  ground  complexes  etudied.  Many  of  the 
observed  differences  were  quite  large.  For  example,  about  twice  as 
many  acquisitions  were  made  by  stationary  ground  vehicles  as  by  fluid 
vehicles,  about  2$  times  as  many  acquisition  advantages  were  scored 

by  stationary  target  vehicles  as  by  moving  targets,  the  mean  interacqui¬ 
sition  advantage  against  scout  elements  was  twice  as  long  for  static  iiriits 
as  for  fluid,  and  over  three  times  as  many  simulated  rounds  were  fired 
by  stationary  vehicles  as  by  moving  vehicles. 

CONCLUSIONS  .  .  Based  on  the  summary  data  presented  in  Tables  2-4 
and  on  the  more  detailed  statistical  analyses  conduced  for  each  effective  • 
ness  measure,  it  was  concluded  that  in  a  winter  environment  against  . 
targets  of  the  type  studied: 

1,  The  low/dismount  tactic  is  more  effective  than  the  tactics  of 
flying  high  or  nap-of-the-earth  with  pop  up, 

2.  The  overall  effectiveness  of  a  platoon  of  M114A1  scout  vehicles 
is  similar  to  that  of  a  single  OH-13  helicopter  employing  the  nap-of-ths 
earth  with  dismount  tactic.  The  ground  scout  platoon  was  mors  effective 
on  the  stationary  runs  and  the  helicopter  dismount  tactic  on  the  moving 
runs. 


3.  The  performance  of  both  helicopters  and  ground  scouts  was 
significantly  better  against  fluid  vehicles  with  a  movement  mission  than 
against  stationary  target  complexes. 

RECOMMENDATIONS,  If  it  is  decided  to  employ  either  a  ground 
scout  platoon  or  helicopters  on  winter-time  reconnaissance  missions  in 
terrain  similar  to  the  type  studied,  it  is  recommended  that: 
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1.  Ground  scouts  be  used  sgainst  suspected  stationary  targets  it  time 
permits.  If  time  does  not  permit,  the  helicopter  tactic  of  low/dismount 

is  suggested. 

2,  In  reconnoitering  against  a  fluid  enemy,  helicopters  using  the 
dismount  tactic  should  be  employed. 
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A  STUDY  OF  PROBABILITY  ASPECTS  OF  A 
SIMULTANEOUS  SHOCK  WAVE  PROBLEM 


A  method  of  solving  probabilistic  problem*  without  a  computer. 

Edward  C.  Hecht 

Picatinny  Arsenal,  Dover,  New  Jer*ey 


I  am  going  to  present  a  procedure  for  the  rapid  solution  by  desk  cal¬ 
culator  of  an  involved  probabilistic  problem.  Figure  1  ia  a  *ample 
computation,  «howing  all  the  paperwork  nece**ary  for  one  solution.  The 
first  two  and  the  last  of  even  these  few  columns  are  identical  for  every 
computation  of  this  sort. 

Unfortunately,  although  characteristically,  the  evolution  of  the  simple 
tool  requires  a  long  explanation.  I  have  made  up  a  problem  as  a  vehicle 
for  the  explanation,  and  I  hope  you  will  bear  with  me  as  I  toil  through  it. 

In  a  certain  classified  ordnance  application,  two  HE  weapons  are  de¬ 
tonated,  and  it  is  a  matter  of  concern  whether  the  shock  waves  from  the 
explosives  arrive  at  a  point  between  them  simultaneously  and  before  the 
occurrence  of  a  particular  event  at  the  Intermediate  point. 

Speaking  generally,  we  have  three  events,  each  with  its  own  distrib¬ 
ution  in  time,  and  we  want  to  find  the  probability  associated  with  certain 
spacing*  and  orders  of  occurrence  of  the  events. 

Calling  the  locations  of  the  two  weapons  and  the  intermediate  point 
A,  C,  and  B,  respectively,  as  illustrated  in  figure  2, 

A _ B _ C 

the  problem  is  to  determine  the  probability  of  arrival  of  shock  waves  from 
A  and  C  at  B  simultaneously  and  before  the  occurrence  of  an  event  at  B 
(called  hereafter  event  B),  Simultaneous  is  defined  arbitrarily  as  within 
100  micro-seconds.  The  expected  times  of  the  detonation  and  event  B  may 
be  the  same  or  different  in  some  ordered  manner. 

For  visualisation  purposes,  it  may  be  considered  that  the  interaction 
effects  of  the  two  shock  fronts  are  to  be  photographed  using  a  Schliersn 
technique,  The  shockwaves  must  meet  within  the  brief  angle  covered 
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by  the  camera.  The  camera's  film  supply  is  limited;  and  event  B  is  the 
start  of  film  exposure.  The  100-microsecond  simultaneity  period  is  the 
time  within  which  the  interaction  is  within  the  narrow  range  of  the  camera. 
This  is  not  the  real  problem,  but  a  hypothetical  problem  that  I  have 
invented,  since  the  true  problem  is  classified.  I  want  to  emphasize  that 
it  is  the  general  method  of  solution,  rather  than  the  problem,  that  I  want 
to  present.  To  make  the  problem  fit  the  solution,  system  failure  must  be 
thought  of  as  coincidence  of  the  shock  waves  at  B  but  before  the  film  has 
started  running. 

This  paper  will  develop  a  procedure  for  determining  the  probability 
of  system  failure  given  the  expected  times  of  the  detonation  of  the  HE 
weapons,  the  expected  time  of  event  B,  and  the  probability  distribution 
of  these  times. 

In  the  situation  in  which  this  problem  arose,  it  was  necessary  to  find 
a  solution  because  the  probability  of  the  shock  waves  arriving  at  B 
simultaneously  and  before  event  B  occurred  was  required  to  be  very 
small,  of  the  order  of  0.  001,  while  the  variability  of  some  cf  the  proposed 
detonators  and  other  components  was  of  the  same  order  as  the  shock 
wave  travel  time  from  A  or  C  to  B.  It  was  necessary  to  find  whether  such 
variability  could  be  tolerated,  and,  if  not,  how  tight  the  dispersion  had  to 
be.  In  order  to  aid  the  required  design  decisions,  it  seemed  desirable  to 
get  the  results  in  the  parameterized  form  of  a  plot  of  system  sigma  versus 
probability  for  selected  shock  wave  travel  times. 

As  a  matter  of  personal  preference,  I  looked  for  a  desk  calculator 
solution,  which  might  later  be  programmed  for  computer. 

In  its  simplest  form,  which  I  will  discuss  first,  the  problem  has  A 
and  C  equidistant  from  B,  so  that  the  shock  wave  travel  times  are  equal, 
and  all  events  will  be  expected  to  be  absolutely  simultaneous. 

The  problem  requires  finding,  for  every  infinite ssimal  interval  of 
time,  the  differential  probability  of  system  failure,  which  is  the  product 
of  three  probabilities  --  the  probability  that  A  detonates  within  that 
infinite  ssimal  interval;  the  probability  that  C  detonates  within  100  micro¬ 
seconds  of  the.  interval;  and  the  probability  that  B  has  not  occurred  one 
shock  wave  travel  .time  later  than  that  interval.  Integrating  the  product 
of  these  probabilities  over  all  time  gives  us  the  total  probability  of 
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system  failure,  For  the  example  used,  the  probability  distribution*  of 
the  event  times  were  all  taken  as  normal  and  the  standard  deviations  as 
equal  for  A  and  C;  but  these  assumptions  are  not  necessary  to  apply  the 
general  method  of  solution. 

Now  I  will  put  the  problem  in  more  general  terms,  In  any  infinitessi- 
mal  period  of  time j  dt,  at  a  time  t ' ,  the  probability  of  system  failure 
is  the  compound  probability  that  event  A  has  happened  a  constant,  pre¬ 
dictable  time  earlier  than  t ' ,  that  event  C  has  happened  within  a  stated 
small  interval  about  a  constant,  predictable  time  earlier  than  t\  and 
that  event  B  has  not  yet  happened  at  time  t 1 .  The  constant,  predictable 
times  are  the  shock  wave  travel  times  from  A  and  C  to  B,  which  may  or 
may  not  be  equal  in  the  general  case;  and  the  stated  small  interval  is  the 
simultaneity  period,  which  must  be  small  enough  relative  to  the  travel 
times  that  events  occurring  within  it  may  be  considered  simultaneous, 

Integrating  this  compound  probability  over  all  time  yields  the  total 
probability  of  system  failure. 

As  illustrated  in  figure  3,  we  will  call  the  times  of  occurrence  of 
events,  A,  B,  and  C ,  tA«  tQ,  and  t^ ,  and  the  probability  distributions 

of  these  events,  P(t^  ?  t),  P(t^  >t),  and  P( t^.  £  t),  The  shock  wave 

travel  time  depends  largely  on  the  travel  distance  and  on  the  amount  of 
explosive  involved,  and  is  considered  to  be  constant,  We  will  call  the 
travel  times  from  A  to  B  and  from  C  to  B,  t^  and  t^g,  The  short 

period  within  which  shock  wave  arrival  is  considered  simultaneous,  we 
will  call  A  . 

Starting  with  the  probability  distributions  of  the  times  of  events 
A,  B,  and  C,  each  of  which  has  somewhat  of  the  appearance  of  the  top 
curve  of  figure  4,  we  proceed  as  follows  to  find  the  probability  of 
system  failure . 

For  convenience  in  notation  and  in  thinking  about  the  problem,  we 
will  tie  our  general  time  frame  to  the  time  frame  of  event  A,  This  poiss 
no  difficulty  since  the  expected  times  of  events  A,  B,  and  C  are  known; 
and  we  would,  in  any  event,  have  used  one  of  these  fixed  times  as  the 
origin  of  the  general  time  system, 
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For  a  system  failure  to  occur  at  time  t',  then,  t'  3  t^  +  t^g,  Alto, 

for  the  necessary  simultaneity,  t^.  must  occur  within  the  period  A  and 

later  than  t^  by  the  difference  between  their  travel  times  to  B;  or,  in 
mathematical  language,  as  it  is  written  on  the  figure  3,  (Of  course,  it 
is  an  algebraic  "later"  and,  if  t^g’t^g  i*  negative  in  sign,  t^  must 

occur  earlier  in  time  than  t .  for  simultaneity  of  shock  wave  arrival  at 

B. ) 

Finally,  for  system  failure,  when  the  simultaneous  shock  wave 
arrives  at  B  at  time  t\  event  B  must  not  have  occurred,  Therefore, 

The  probability  that  event  A  occurs  within  any  differential  period  of 
time,  dt,  is  d  P  (t^  £  t),  This  differential  probability  must  be  multiplied 

by  the  probability  that  event  C  occurs  in  an  interval,  Ai  ^at#r 

than  dt,  P  (tA  +  tAB  •  tCB  -  A  <  >c  ‘  'a  +  'AB  ’  ‘CB  *  L  >' 

The  product  must  further  be  multiplied  by  the  probability  that  event 
B  occurs  after  t\  P  (tg  >  t^  +  tAg).  Integrating  this  final  product  over 

all  P  (tA  Sf  t)  is  equivalent  to  integrating  over  all  t,  since  P  (t^  >  t)  is 

a  single  valued  function  of  t. 

The  probability  that  C  occurs  within  the  simultaneity  interval  of  any 
time  t  is  obtained  from  the  probability  distribution  of  the  times  of  event 

C,  In  the  case  of  normal  distributions,  this  is  easy  to  do, 

The  curve  of  this  function  versus  t  has  the  general  form  of  the  bot- 
tom  curve  of  figure  4,  Then  the  probability  distribution,  P  (tg  >  t)  versus 

t  is  modified  to  P  (tfi  -  tAg  2  t)  versus  t. 

These  two  functions  of  t  are  multiplied  together  to  get  P  (tg  -  t^g^t), 
F  ^  +  *AB  “  *CB  <  *C  <  *  +  *AB 


t^B  +  A  )  versus  t, 


Since  P  (t ,  t)  is  a  single  valued  function  of  t,  values  of  this  proba- 

A 

bility  can  be  subetitued  for  values  of  t  to  get  a  plot  of  P  (tg  -  t^g  >  t) 
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time*  P  1 1  +  t  .  _  -  t _ -A  <  t  <t+t.  -  t_  .  »  A  )  «wr«bi  Pit  tl 

-n.o  tc  u  AJS  v«B  '  A"" 

(ae  in  figure  5).  The  area  under  this  lart  curve  is; 


^lp(ta  5 '  *  lAB>  p<t  *  'ab-'cb  'h<tc  <  ‘  *  *AB  -  >CB  ♦  *  >  " 


and  thi«  is  the  probability  of  a  lystem  failure,  P^.  The  important 
attribute  of  this  method  of  solution  is  that  this  area  may  readily  be 
evaluated,  without  constructing  the  curves,  by  use  of  Simpson's  Rule. 

If  a  Simpson  Rule  division  of  the  area  into  ten  parts  gives  us  enough 
accuracy  (as  it  well  may,  depending  on  how  accurately  we  know  the 
shapes  of  the  distributions  involved),  we  need  only  find  eleven  value*  of 
t  corresponding  to  P  (tA  ^t)  values  of  0,  0, 1,  0,  2,  etc.  ,  to  1.  0;  and  two 

of  these  times  are  plus  and  minus  infinity.  At  these  extreme  times,  the 
simultaneity  probabilities  are  taro.  For  the  intermediate  times,  the 
simultaneity  probability  may  easily  be  looked  up  in  any  well-detailed 
table  of  areas  under  the  normal  curve  for  the  normal  distributiona 
assumed  in  our  example. 

A  sample  computation  has  been  shown  in  figure  1,  For  any  computa¬ 
tion  using  a  10-part  Simpson  Rule  integration,  the  first  two  columns  will 
be  the  same.  To  get  the  simultaneity  probabilities,  it  is  observed  that 
A  s  0,  01  •  so  that  for  the  second  time  point  the  probability  looked 

up  is  that  of  being  between  1.  292  andL  272  standard  deviations  away 

from  the  mean.  For  the  column  of  t  +  t  _  ,  it  it  noted  that  t  ■  In’- 

Ad  Ao  B 

and  that  c.  =  2?  .  Then  since  t  .  *  7  ,  7,  +  kr  =  t  +  (2k  +  l)r  ....  . 

Having  found  these  t^  equivalents,  the  table  look-up  is  easy,  The  next 

column  is  the  product  of  the  third  and  fifth  columns.  These  products  are 
multiplied  by  the  Simpson  Rule  factor*,  1,  4,  2,  4,  2,  etc,  ;  and  the  sum 
is  multiplied  by  the  clasa  interval  of  0.1  and  divided  by  3  to  get  the  value 
of  the  integral  (see  figure  4),  which  is  the  answer  sought.  Repetition 
enables  the  computation  to  be  performed  in  about  20  minutes. 

Taking  expected  times  as  equal,  and  at  least  two  of  the  sigmas  as 
equal,  allows  results  to  be  plotted  as  in  figure  6.  Other  conditions  are 
not  much  harder  to  compute,  but  the  resuite  are  harder  to  present, 
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Many  problems  besides  this  fictionalized  and  hypothetical  one  arc 
susceptible  to  this  technique  of  solution,  The  method  is  one  which,  with 
some  familiarity,  enables  an  engineer  or  mathematician  to  solve  problems 
involving  probability  at  hie  desk  before  or  without  submitting  them  for 
computer  solution,  And,  of  course,  it  is  useful  for  those  who  have  no 
access  to  a  computer, 
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FIGURE  1 
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A  DATA  COLLECTION  PROCEDURE  FOR  ASSESSING  NEURO-MOTOR 
PERFORMANCE  IN  THE  PRESENCE  OF  MISSILE  WOUNDS 

William  H,  Kirby,  Jr.,  William  Kokinakis , 

Larry  M,  Sturdivan,  and  William  P.  Johnson  - 
U.  S.  Army  Ballistic  Research  Laboratorie s 
Aberdeen  Proving  Ground,  Maryland 


INTRODUCTION,  While  medical  clinicians  diagnose,  treat,  and  judge 
the  sequels  of  injury,  it  is  of  interest  to  others  such  as  those  engaged  in 
man-task  or  man-machine  system  studies  to  consider  the  effects  of  injury 
from  additional  points  of  view.  Those  concerned  with  the  medical  problems 
are  naturally  interested  in  procedures  and  optimal  treatments  which  pre¬ 
vent  or  at  least  minimize  the  consequences  of  injury.  Those  concerned 
witn  man-machine  system  performance  problems  are  interested,  in  addi¬ 
tion,  in  the  ability  of  the  injured  or  otherwise  stressed  individual  to  per¬ 
form  given  tasks,  Common  to  both  the  clinician  and  the  man-machine 
researcher  interested  in  injury  is  a  need  for  a  better  understanding  of 
the  mechanisms  and  responses  associated  with  traumatic  pathological 
dynamic  s . 

We  are  in  the  process  of  developing  a  methodology  for  describing  and 
assessing  anatomical  and  physiological  pathologies  associated  with  missile 
wounds,  Hopefully,  we  will  be  able  to  express  these  in  terms  of  a  set 
of  affectors  and/or  effectors  such  as  those  found  in  the  nerve-muscle  or 
neuro-motor  structure,  The  reason  for  this  approach  is  that  they  may 
serve  as  a  common  denominator  for  describing  injury  as  well  as  for 
describing  the  task  or  machine  operation  requirement.  In  this  presenta¬ 
tion  wa  will  limit  our  attention  to  wounds  caused  by  missiles.  The  type 
and  amount  of  data  to  be  collected  will  probably  be  influenced  by  the 
number  of  accident  cases  that  enter  hospitals  which  are  accessible  for 
study. 

This  is  an  interdisciplinary  problem  area  in  which  clinicians,  engi¬ 
neers,  mathematicians  and  statisticians  should  meet,  It  is  usually  the 
case  that  such  multi-discipline  representatives  are  faced  early  in  the 
process  with  certain  communication  problems.  For  example,  a  function 
to  the  clinician  has  one  meaning,  but  to  the  mathematician  it  has  quite 
another,  A  medical  researcher  may  apply  a  Chi-Square  test  to  a  set  of 
data  in  which  the  statistician  may  insist  that  the  application  is  invalid  due 
to  the  fact  that  the  data  do  not  conform  to  a  normal  distribution,  A  surgeon 
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rnay  be  entirely  satisfied  that  the  maximal  strength  of  a  grasp  is  equal  in 
both  hands  of  a  patient  as  determined  through  a  hand  squeezing  process 
whereas  the  engineer  is  satisfied  only  if  such  an  assessment  is  in 
quantitative  terms  such  as  a  pressure-time  history.  Clinicians  can 
really  get  confused  whenthey  attempt  to  understand  differences  between 
mathematician^  and  statisticians. 

DISCUSSION.  Our  first  approach  considers  the  body  as  system 
composed  of  a  set  of  clinical  subsystems  coordinated  to  maintain  life 
and  control  human  performance.  These  clinical  subsystems  will  initial3y 
be  divided  for  convenience  into  a  primary  and  secondary  group.  The 
primary  group  will  include  the  neurological,  cardiovascular,  respiratory, 
skeletal,  and  muscular  sybsystems.  The  secondary  group  will  consist 
of  the  gastrointestinal,  genitourinary,  and  endocrine  subsystems.  While 
we  intend  to  collect  some  data  associated  with  the  secondary  group, 
initially  only  the  primary  group  will  be  considered  in  detail. 

We  attempt  to  describe  performance  in  terms  of  a  simplified  set  of 
afferent  and  efferent  (input-output)  factors  shown  in  Figure  1.  For  the 
present  we  intend  only  to  recognize  the  presence  or  absence  of  the 
afferent  (input)  factors  -  vision  and  hearing,  and  the  efferent  (output) 
factor  -  voice.  Essentially  then  we  have  reduced  our  performance  descrip¬ 
tors  to  the  first  six  listed  in  Figure  1.  Actually  these  descriptors  are 
regional  subdivisions  of  the  human  body  and  they  will  be  represented  by  the 
(motor)  muscles  which  are  located  in  the  respective  regions.  The  neurol¬ 
ogical  or  muscle  activator  network  is  distributed  over  these  regions  and 
no  controlled  human  actions  occur  without  its  activation.  It  was  therefore 
natural  to  choose  these  motor  factors  as  a  common  denominator  to  which 
all  performance  phenomena  and  subsystem  changes  may  be  related. 
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Using  the  above  rationale  we  are  interested  in  collecting  data  from 
accidental  wound  cases  in  order  to  describe,  classify,  and  relate 
important  missile  characteristics,  clinical  subsystem  injuries,  and 
neuro-motoi  performance  phenomena.  Concurrently  a  more  comprehen¬ 
sive  study  of  this  type  of  problem  is  being  considered  but  which  is  beyond 
the  scope  of  this  presentation. 

The  Neuro-Motor  or  Effector  Logic.  The  "terminal"  body  tissue  or 
structure  directly  responsible  for  physical  movements  as  indicated  above 
is  muscle.  Inasmuch  as  muscles  are  innervated  by  specific  peripheral 
nerves,  associated  nerves  and  muscle*  have  become  known  as  "neuro- 
motor  units.  "  Fortunately  the  nerve -muscle  anatomical  distribution 
system  has  been  well  established  by  anatomists  in  the  past. 

In  order  to  demonstrate  this  logic  attention  is  directed  to  Figure  2 
which  is  a  matrix  showing  the  muscles  and  their  actions  in  the  upper 
limb.  This  matrix  could  represent  either  of  the  effectors,  or  e^ 
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since  they  aie  symmetrical,  The  numbers  along  the  abscissa  are  sub¬ 
scripts  of  the  letter  "A"  in  which  each  subscript  represents  a  specific 
anatomical  action  as  described  in  Appendix  A.  The  numbers  along  the 
ordinate  are  subscripts  of  the  letter  "M"  in  which  each  Jubscript 
represents  a  specific  muscle  aiso  described  in  Appendix  A.  The  muscles 
(m.,  i  =  l ,  Z ,  .  .  .  61)  are  arranged  in  a  manner  such  that  the  lower  numbers 
represent  muscles  in  the  shoulder  and  in  ascending  order  represent 
muscles  in  the  arm,  forearm,  and  hand. 

The  distribution  of  nerves  and  their  contained  fibers  which  innervate 
the  skeletal  muscles  is  unique,  For  instance,  the  large  number  of  nerve 
fibers  which  originate  from  a  given  source  such  as  a  particular  spinal 
cord  segment,  are  dispersed  into  a  multiplicity  of  branches.  As  if  to 
provide  maximum  reliability,  many  fibers  from  the  same  source  reach 
a  given  muscle  by  different  pathways,  On  the  other  hand  nerve  fibers 
from  the  same  spinal  cord  level  are  known  to  innervate  different  muscle*. 
While  the  nerve  pathways  are  not  demonstrated  here,  some  idea  of  the 
nerve  fiber  distribution  may  be  obtained  from  the  left  side  of  Figure  3. 
The  C,  (i  =  1,  2,  ,  .  ,  8) ,  T^,  and  T^  represent  nerve  roots  (large  groups 

of  fibers)  which  emerge  from  the  designated  spinal  cord  levels,  These 
are  identified  in  Appendix  A.  The  letter,  C,  refers  to  the  cervical  or 
neck  region  and  the  subscripts  refer  to  the  specific  locations  out  of 
which  the  bundles  of  nerve  fibers  flow,  The  letter,  T,  refers  to  the 
thoracic  or  chest  region,  Only  the  first  two  thoracic  nerve  bundles, 

T^  and  T^,  are  included  in  Figure  3, 

It  is  interesting  to  observe  from  this  matrix  that  a  given  muscle 
is  innervated  by  nerve  fibers  from  more  than  one  source.  Note, for 
example,  that  (pectoralis  major)  is  innervated  by  nerve  fibers 

derived  from  several  spinal  cord  segments,  namely,  Cg,  C^,  C^,  Cg, 

and  T  .  An  important  implication  in  conjunction  with  the  previous  com¬ 
ments  on  reliability  is  that  if  the  spinal  cord  were  severely  injured  at 
the  level  of  C  ,  muscle,  M  ,  would  not  become  completely  paralysed 

inasmuch  as  it  would  still  receive  considerable  innervation  from  fibers 
above  the  site  of  injury  namely,  C.,  C^,  and  C^,  I*  is  also  interesting 

to  observe  the  number  of  muscles  in  the  shoulder  region  innervated  by 

a  given  nerve  root  such  as  C  .,  They  include  M_  ,  M„,  M,,  M_,  M, . , 

°  5  2  3  4  7  10 

^"^11*  ^12  ’  ^^1 3 1  5  >  y  ,  Ni^g,  ^19' 
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66  REPRESENT  ANATOMICAL  ACTIONS 
REPRESENT  UPPER  LIMB  MUSCLES 


FIGURE  2 
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This  multiplicity  of  overlap  or  redundancy  is  unique  for  minimizing 
the  effects  nf  injury  Quantitative  force  time  relationships  hSvo  uul  been 
effectively  established  at  the  muscle  level  to  allow  us  to  assign  weighting 
factors  to  the  contributions  from  a  muscle  to  an  associated  anatomical 
action. 

Single  muscles  and  their  respective  anatomical  actions  combine  to 
characterize  regional  and  joint  actions,  In  the  shoulder  region,  tor 
instance,  we  think  first  in  terms  of  the  muscles  associated  with  the 
specific  anatomical  actions  such  as  flexion,  extension,  abduction,  and 
adduction,  In  continuing  the  generalization  process  in  this  region  we  next 
consider  the  shoulder  motions  oi  rotation  and  circumflexion  which  are 
derived  from  the  same  muscles  acting  in  different  co  mbinations  and 
sequences,  In  applying  the  same  notion*  to  the  hand  we  begin  by  consid¬ 
ering  the  single  muscles  associated  with  finger  flexion,  extension, 
abduction,  adduction  and  opposition  and  then  combining  these  in  ways  to 
Account  for  generalized  processes  such  as  graspitig,  holding,  and  releas¬ 
ing,  They,  of  course,  are  associated  with  even  more  complicated 
processes  associated  with  performing  tasks  such  as  using  a  screw 
driver  or  turning  a  door  knob. 

In  brief  we  hope  to  be  able  to  aseociate  the  biomechanical  functions 
with  the  natural  effectors  or  neuro-motor  factors  which  are  responsible 
for  them.  One  may  proceed  in  man-task  study  problems  in  either  direc¬ 
tion,  i,  e.  ,  he  may  begin  with  the  knowledge  of  basic  muscular  functions 
and  move  up  the  scale  to  gross  movements  or  he  may  start  with  a  study 
of  the  man-task  process  in  the  hope  of  first  identifying  useful  grose  move¬ 
ments  and  work  down  to  the  scale  to  muscle  functione, 

A  combined  mechanical  and  anatomical  orientation  appear*  to  have 
some  unique  advantages  for  describing  man-machine  interaction,  For 
example,  we  believe  that  by  considering  the  upper  limb  a*  a  flexible 
multi-jointed  cantilever  with  a  unique  prsheneile  of  grasping  device 
located  at  the  free  end,  one  can  develop  useful  methods  for  describing 
physical  and  physiological  factors  in  relation  to  man-machine  inter¬ 
actions  in  ways  which  yield  to  simplification  and  me&sursment, 

In  our  first  approach  to  upper  limb  biomechanical  measurement*  we 
are  ueing  these  anatomical-mechanical  notions,  i,e.  ,  muscle  groups 
associated  with  hand  actions,  joint  actions,  multi-joint  actions,  liner 
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actions,  and  combinations  of  these.  Initially  we  propose  to  measure  only 
a  limited  number  of  these  biomechanical  or  effector  functions.  For  the 
upper  limb  we  have  developed  some  instrumentation  to  measure  and 
record  force-time  histories  associated  with  hand  grasping,  and  flexion 
and  extension  actions  about  the  wrist  and  elbow  joints.  A  brief  explana¬ 
tion  of  this  instrumentation  is  given  in  Appendix  B, 

We  extend  this  rationale  to  include  the  opposite  upper  limb,  the  two 
lower  limbs  (considering  them  also  as  multi-jointed  cantilevers  but  in 
terms  of  their  natural  anatomical-mechanical  functions  of  weight-bearing 
and  ambulation),  and  the  other  effectors  (head  and  neck,  ana  trunk).  We 
believe  that  this  approach  will  result  in  useful  descriptors  for  man-task/ 
machine  interactions  in  a  manner  suitable  for  describing  and  assessing 
changes  in  performance  due  to  disability  regardless  of  cause. 

CLINICAL  SUBSYSTEMS.  One  of  the  reasons  this  problem  is  of 
interest  is  that  it  requires  investigations  not  made  in  the  past.  For 
example,  it  is  known  that  damage  to  the  cardiovascular  subsystem  in 
terms  of  blood  loss  is  likely  to  be  fatal  if  the  value  exceeds  approximately 
1600  cc  to  1700  cc  within  a  short  period  assuming  no  replacement.  While 
this  is  an  important  upper  bound,  the  effect  on  one's  ability  to  perform 
due  to  hemorrhage  of  lower  orders  has  not  to  our  knowledge  been  studied. 
Hence  it  is  of  primary  interest  for  us  to  collect  hospital  data  on  patients 
who  may  suffer  various  degrees  of  blood  loss  and  to  measure  the  effects 
on  several  representative  effectors. 

Damage  to  the  respiratory  subsystem  may  be  assessed  in  terms  of 
the  degree  of  pneumothorax,  rate  °f  V  C°2  5  xchange,  or,  perhaps,  In 
terms  of  respiration  rate  and  depth.  The  chosen  effectors  would  be 
measured  at  approximately  the  same  time  that  the  physiological  measures 
are  taken. 

Descriptions  of  levels  of  damage  may  be  more  difficult  for  the 
neurological,  muscular,  and  skeletal  subsystem*.  Presently  we  are 
only  considering  two  level*  of  damage  for  any  substructure  (a  muscle, 
a  nerve,  or  a  bone),  namely,  none  or  complete.  For  the  present  we  do 
not  expect  to  make  special  studies  on  the  gastro-inte stinal ,  genitourinary 
and  endocrine  subsystems  other  than  to  observe  the  routine  hospital  events 
associated  with  them.  Their  respective  blood  losses  are  to  be  considered, 
however,  but  viewed  as  cardiovascular  subsystem  deficits, 
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Additional  measurements  which  are  not  always  routine  may  be  added 
if  the  results  nf  some  of  the  present  research  being  done  elsewhere  on 
traumatic  injuries  indicates  it.  For  example,  one  of  us  is  engaged  in 
human  shock  research  in  which  certain  relations  between  clotting  time  and 
levels  of  shock  have  been  generated  for  30-odd  humans  who  were  in  shock 
due  to  blood  loss.  Interesting  observations  of  adrenal  function  in 
combat  and  wounded  soldiers  have  been  made  to  some  extent  by  others. 

I4)  (5)  These  suggest  possibilities  for  associating  "stress"  levels  and 
performance. 

It  is  with  these  ideas  in  mind  and  the  notion  that  feasible  relations 
between  the  effectors  and  the  body's  clinical  subsystems  do  exist  that  we 
wish  to  construct  useful  data  collecting  procedures.  Hopefully,  early 
insight  following  some  of  this  data  collection  will  allow  us  to  get  some 
ideas  concerning  these  relations.  Since  we  expect  such  potential  relation¬ 
ships  to  change  with  time  as  a  result  of  injury,  we  believe  that  data 
collection  will  have  to  be  made  at  various  time  periods  throughout  the 
clinical  course. 

Having  set  up  these  ideas  as  guidelines  for  the  data  collection  pro¬ 
cess,  we  must  consider  some  of  the  practical  aspects  of  the  problem. 

It  is  important  to  review  the  clinical  procedures  used  In  evaluating  and 
treating  wound  cases  in  hospital  accident  rooms,  operating  room*,  and 
recovery  wards  in  order  to  appraise  the  available  and/or  recorded  data 
in  terms  of  type  and  quantity.  Another  point  of  interest  concerns  the 
logic  for  selecting  and  running  certain  clinical  tests  and  not  others. 

There  may  be  many  cases  in  which  certain  useful  clinical  data  could  be 
made  available  for  specific  purposes  such  as  ours  but  which  msy  not  bo 
sought  ordinarily  by  a  clinician  inasmuch  as  these  data  do  not  in  his 
judgment  add  any  useful  information  for  his  purposes,  It  is  alio  Impor¬ 
tant  to  be  sure  that  the  acquisition  of  data  from  a  distressed  patient  does 
not  interfere  with  his  well-being. 

Medical  Records  and  Clinical  Data.  Hospital  medical  records 
reflect  traditional  procedures  for  recording  information  and  events 
associated  with  the  professional  care  and  treatment  of  patients,  While 
time  and  space  preclude  any  extensive  discussion  of  the  meaningfulne si 
of  the  comprehensive  clinical  and  laboratory  data  as  interpreted  by 
physicians  and/or  other  interested  discipline  representatives,  a  few 
observations  are  presented, 
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Clinicians  evaluate  patients,  their  care  and  theraphy,  according  to 
the  description  and  history  of  the  presenting  complaint  as  well  as  other 
pertinent  past  patient  (and  family)  history,  physical  examination, 
laboratory  test  results,  and  progress  evaluations.  In  general  clinical 
information  is  classified  either  as  subjective  or  objective  information. 
Subjective  information  is  associated  with  what  the  patient  or  others 
tell  to  the  examiner.  Examples  of  this  would  include  the  patient's 
interpretation  of  local  or  general  muscular  weakness,  walking  difficulty, 
- dragging  toe  of  shoe,  stumbling  or  falling,  sphinteric  disturb¬ 
ances  (inability  to  hold  urine),  changes  in  local  or  general  sensation  ---- 
fixed  or  radiating  pain,  temperature,  tactile  discrimination,  deep  sensa¬ 
tion  (muscle,  bone,  vibratory  sense),  and  abnormal  sweating.  Objective 
information  concerns  what  the  examiner  learns  from  his  own  observations 
such  as  range  of  limb  movement,  contractures,  diminished  size,  strength 
of  muscles  against  resistance,  tremors,  etc.  Thus,  it  can  be  seen  that 
except  for  laboratory  tests  and  some  clinical  items  such  as  blood  pres¬ 
sure,  pulse  rate,  and  the  electrocardiogram,  quantitative  measures 
are  minimal  in  the  traditional  records. 

Records  of  emergency  cases  are  often  initiated  in  the  hospital's 
accident  room  and  accompany  the  patient  throughout  his  hospital  course. 

A  form  of  time  history  of  his  care,  diagnosis,  treatment,  and  progress 
is  recorded  for  permanent  file.  We  will  consider  a  few  examples  of 
hospital  records. 

An  accident  room  record  is  shown  in  Figure  4.  This  21  year  old 
male's  record  shows  very  little  information.  This  is  not  unusual  in  the 
typical  busy  accident  room.  The  only  history  and  physical  data  recorded 
in  this  case  are  blood  pressure  and  pulse.  The  immediate  treatments 
and  the  results  of  laboratory  blood  tests  were  recorded.  It  is  highly 
likely  that  additional  observations  of  blood  pressure  and  pulse  were 
made  with  the  passage  of  time  but  not  entered  in  the  record.  A  certain 
amount  of  physical  examination  was  probably  performed  and  not  recorded. 
While  the  admission  time  and  date  were  recorded,  a  detailed  history  of 
the  presenting  complaint  is  not  shown.  Such  information  might  have 
included  approximate  time  of  the  shooting  incident,  estimates  of  external 
blood  loss,  and  conscious  or  unconscious  behavior  of  the  patient  until 
arrival  at  the  hospital,  and  ballistic  factors,  e.  g.  ,  type  and  calibre  of 
gun,  distance  between  firer  and  patient,  and  angle  of  target  to  firer. 
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Another  accident  room  record  is  shown  in  Figure  5.  This  15  year 
old  male  with  multiple  gun  shot  wounds  has  much  more  information  on 
the  physical  examination  than  the  previous  case.  Another  example  is 
shown  in  Figure  6.  One  must  judge  that  in  this  latter  case  the  patient 
was  moved  without  delay  to  a  ward  bed  for  further  control  and  treatment. 

In  examining  a  number  of  similar  records,  we  believe  that  informa¬ 
tion  which  is  useful  for  the  clinician  is,  for  our  purposes,  both  inexact 
and  insufficient.  It  is  g^ot  enough  for  us  to  know  that  "the  neurological 
and  extremities  are  negative.  "  It  is  our  opinion  that  we  need  to  have 
some  measured  data  associated  with  the  neuro -motor  activity  of  the 
effectors.  For  example,  the  prehensile  or  grasping  function  of  an 
upper  limb  although  not  injured  may  be  weakened  due  to  a  severe  blood 
loss  in  an  artery  located  in  another  part  of  the  body.  Neurologists  and 
neurosurgeons  have  a  variety  of  clinical  tests  which  they  use  based  on 
certain  known  relationships  in  neuro-anatomy  and  physiology.  As  a 
matter  of  interest,  it  has  been  stated  that  in  no  other  branch  of  medicine 
is  it  possible  to  build  up  a  clinical  picture  so  exact  as  to  localization 
of  pathological  conditions.  (2)  However,  if  such  tests  are  not  performed 
and/or  the  results  not  recorded  for  patients  of  interest  to  uc ,  possible 
relations  between  the  effectors  and  clinical  subsystem  changes  are  not 
attainable. 

It  may  be  noted  also  that  many  classical  neuro -motor  tests  are 
not  strictly  quantitative  as  evidence  by  such  observational  descriptions 
as  "partial  paralysis"  or  "some  loss  of  sensation.  "  Conclusions 
drawn  from  a  variety  of  tests  may  vary  considerable  as  they  depend 
on  the  judgment  of  different  examiners  and  the  impressions  expressed 
by  patients. 

In  the  ward  the  situation  is  quite  different  from  that  found  in  the 
accident  room.  Here  the  patient  is  in  an  environment  in  which  physi¬ 
cians,  nurses,  and  technicians  are  available  to  care  and  control  the 
patient  under  more  favorable  conditions.  Records  are  kept  in  much 
more  detail.  The  frequency  of  observations  and  the  application  of 
procedures  and  treatments  are,  of  course,  greater  when  the  patient  is 
in  serious  condition.  Except  for  emergency  procedures,  these  observa¬ 
tions  and  treatments  for  patients  under  intensive  care  are  usually  per¬ 
formed  about  every  one  of  two  hours.  It  takes  this  long  for  many 
therapies  to  take  effect. 
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Whenever  a  patient  undergoes  a  special  procedure,  operation,  or, 
if  an  autopsy  is  performed,  a  detailed  account  of  the  events  is  made  which 
becomes  a  part  of  the  permanent  medical  record.  The  first  detailed 
physical  examination  is  often  not  performed  until  the  patient  arrives  in 
the  ward  following  evaluation  and  treatment  in  the  accident  room,  Addi¬ 
tional  laboratory  tests  and  special  diagnostic  procedures  are  usually 
initiated  after  admission  to  the  ward,  A  considerable  amount  of  clinical 
information  is  recorded  which  can  be  useful  for  our  purposes,  Large 
gaps  invariably  exist,  however,  due  to  reasons  mentioned  previously  as 
well  as  the  thoroughness  of  the  work  and  recording  of  interne  and  resi¬ 
dents.  Patient  care  is  naturally  oriented  toward  healing  and  care  pro¬ 
cesses.  This  is  to  say  that  attention  is  on  the  progress  of  ths  patient's 
subsystems  and  behavior  as  a  whole  while  he  is  in  a  resting  state.  He 
is  not  considered  as  a  component  in  a  man-machine  system. 

A  Given  Wound  Patient-  Record.  In  order  to  get  an  idea  of  some  of 
the  events  which  may  take  place  in  an  accident  room  and  a  ward,  a 
medical  record  of  a  patient  admitted  with  a  bullet  would  is  briefly 
discussed,  .  < 

Accident  Room. 

J  . 

1st.  Day:  A  56  year  old  man  was  shot  in  the  chest  with  a  32  Calibre 
pistol  "at  close  range.  "  He  was  taken  to  a  hospital  and  arrived  at 
5;  36  a,  m.  on  the  day  of  the  accident. 

On  admission  to  the  accident  room,  the  following  were  studied  or 
measured  immediately  and  recorded! 

Blood  Pressure 
Pulse  rate 
Heart  sounds 
Breath  sounds 
Hemoglobin 
Chest  X-ray 

Treatment  was  also  ordered  and  initiated  immediately.  It  consisted 
of  a  fluid  replacement  program  according  to  the  following  sequence: 
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Circulatory  expander  started  (500  cc) 

Whole  blood  (500  cc) 

Saline  (500  cc) 

Dextrose  in  water  (1000  cc) 

The  patient  was  also  sedated  and  prevented  from  taking  any  fluid  or  food 
by  mouth  (as  a  precaution  in  case  of  the  need  for  operation), 

The  remaining  accident  room  events  which  occurred  according  to 
the  medical  record  were: 

Follow-up  blood  pressure,  pulse  rate,  and  respiration  checks  at 
8:  00  am,  8:  25  am,  9:  00  am,  10:  00  am,  12:  00  noon,  4:  00  pm  and  6;  00  pm, 

The  patient  was  admitted  to  the  ward  at  6:  30  pm, 

Ward  The  admitting  intern  made  the  first  detailed  Investigation  at 
7:  30  pm.  It  is  shown  as  follows: 

Intern's  Admission  Note:  "At  4:  00  am  patient  was  shot  by  wife  at 
closo  range  with  a  ,  32  Calibre  pistol,  " 

Bullet  entered  left  chest  and  escaped  via  left  back. 

Patient  did  not  lose  consciousness, 

There  was  no  chest  pain,  coughing  increase,  or  hemoptysis  (coughing 
of  blood). 

Physical  Examination:  Well  developed, 'well  nourished,  alert, 
cooperative,  no  distress  (two?)  gun  shot  wounds,  no  powder  burns, 

Entrance:  approx,  5  mm  left  anterior  axillary  line ,  8th  interspace 

Exit:  Post  axillary  line,  10th  interspace 

Chest:  Expands  well  bilaterally 

Lungs  -  left  posterior;  basilar  dullness,  about  3  cm  above 
the  base;  tactile  fremitus  (a  vibration  imparted  to  the  hand 
placed  on  the  chest)  somewhat  impaired  left  posterior  base. 
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Blood  r-resoure:  iZu/60  -  no  murmurs 

Abdomen:  Slightly  distended 

Generalized  superficial  tenderness 

Neurological  and  Extremities  -  negative 

Impressions:  (1)  gunshot  wound,  left  chest 

(2)  left  hemothorax 

(3)  rule  out  perforated  bowel  or  spleen 

(4)  Cardiac  arrythmia  -  bigemini  (paired  pulse  beats) 

Additional  medications  were  given  at  8:  00  pm. 

2nd,  Day:  Throughout  the  second  day  the  following  vital  signs  were 
observed  every  hour. 

Blood  pressure 
Pulse  rate 
Respiration  rate 
Temperature 

Nourishment  was  still  given  by  intravenous  fluids. 

3rd.  Day:  Vital  signs  were  observed  every  four  hours  instead  of 
the  hourly  schedule  of  the  previous  day,  A  liquid  diet  was  prescribed  in 
place  of  the  intravenous  fluids. 

After  three  more  days  the  patient  was  placed  on  routine  care.  Except 
for  a  thoracentisis  (extraction  of  fluid  from  the  chest  cavity  via  needle 
and  syringe)  on  the  8th,  hospital  day  (250  cc  of  bloody  fluid  was  removed), 
the  patient  recovered  and  was  discharged  on  the  12th.  day. 

Inasmuch  as  no  operating  room  procedures  were  required,  there 
was  no  opportunity  to  get  a  pathological  description  of  the  internal  path 
of  the  bullet.  In  operative  cases  wound  tract  descriptions  are  usually 
available  in  varying  amounts  of  detail.  In  many  instances  some  eetimates 
of  hematoma  (tiapped  blood)  volume,  degree  of  bone  fracture,  and  other 
gross  abnormalities  are  noted  in  the  operating  room  reports.  In  some 
cases,  missiles  or  missile  fragments  are  removed  whereas  in  others 
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the  additional  risk  associated  with  the  removal  process  is  such  that  the 
fragment(s)  are  left  in  the  body,  X-ray  studies,  of  course,  are  informa¬ 
tive  in  such  cases. 


In  this  case  the  calibre  of  the  gun  was  known  and  some  indication 
of  distance  between  gun  and  target  was  given.  Occasionally  additional 
information  is  given  which  is  quite  important  such  as  an  account  of  the 
patient's  response  following  the  wounding  process  (e.g.  ,  "the  patient 
ran  to  the  doorway  and  down  the  steps  before  fainting,  "  "the  patient 
screamed  and  fell  to  the  floor  unconscious,  "  etc. ).  Estimates  of 
blood  loss,  if  noticed,  and  elapsed  time  between  the  shooting  incident 
and  arrival  at  the  hospital  accident  room  are  of  basic  interest  to  us. 

From  a  cursory  review  of  medical  records  concerning  missile 
wounds,  one  is  impressed  with  the  fact  that  the  large  majority  of  such 
cases  are  non-lethal.  In  survival  records  it  is  observed  that  patient 
handling,  treatments  and  control  vary  considerably  depending  on  the 
case.  However,  the  general  care  patterns  do  not  appear  to  be  radically 
different.  The  type  and  amount  of  data  recorded  on  the  other  hand 
varies  widely.  Lethal  cases,  of  course,  usually  have  an  autopsy  report 
providing  one  was  permitted  by  next  of  kin  or  ordered  by  a  medical 
examiner. 


Remarks  on  Hospital  Medical  Records.  In  brief  several  points 
concerning  information  and  medical  records  for  patients  entering  the 
hospital  with  gunshot  wounds  are  presented. 

1.  There  is  considerable  variation  in  type  and  content  of 
reported  information.  This  is  particularly  so  in  the  accident  room 
portion  of  the  record. 

2.  While  it  may  be  assumed  that  a  multitude  of  observations 
on  a  patient's  subsystems ,  appearance,  manner  of  behavior,  speech, 
etc.  ,  are  made  by  clinicians  which  may  influence  diagnosis,  treatment, 
and  control  decisions,  such  information  is  not  usually  so  extensively 
recorded. 


3.  Patient  information  is  difficult  to  handle  and  retrieve 
inasmuch  as  it  is  not  organized  according  to  subsystem  variables  over 
time.  Some  of  the  measures  which  are  considered  as  standard  include 
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vital  signs  such  as  temperature,  blood  pressure,  pulse  rate,  and  respira 
tion  rate.  These  are  usually  plotted  on  graph  paper  by  the  nurses.  Clini 
cal  descriptions  and  observations  are  often  matters  of  judgment  which  do 
not  yield  to  measurement  under  present  state-of-the-art. 

4.  Facts  and  observations  pertaining  to  shooting  incidents  and 
especially  anatomical,  physiological,  and  psychological  events  which 
occur  between  the  incident  and  arrival  at  the  hospital  are  usually  very 
brief  when  included. 

5.  What  is  or  is  not  recorded  probably  depends  on  the  amount 
of  training,  experience,  and  judgment  of  the  respective  hospital  staffs 
and  residents. 

6.  Usually  no  psychological  or  psychiatric  examinations  are 
made  on  patients  suffering  from  missile  trauma.  However,  this  is  not 
done  as  a  routine  on  traumatized  patients  in  general. 

Proposed  Data  Collection.  We  have  described  what  we  wish  to  do, 
a  method  for  going  about  it,  the  type  of  data  we  think  we  need  as  a  first 
approximation,  and  the  entent  of  its  awdlability  in  the  (emergency)  hospi¬ 
tal.  It  is  apparent  that  our  requirements  call  for  more  complete  and 
more  frequent  clinical  observations  and  measures  in  addition  to  the  new 
information  in  support  of  our  special  interests.  This  information  may 
be  thought  of  as; 

1.  Ballistic  information  (Appendix  C). 

2.  Postwounding  behavioral  information  (Appendix  D). 

3.  Pathological  information  in  terms  of 

a.  Wound  tract  information  (Appendix  E). 

b.  Clinical  subsystem  information  (Appendix  F). 

4.  Effector  measurement  information  (Appendix  G). 

5.  Special  studies  for 

a.  Information  on  normals  (Appendix  H). 

b.  Anthropometric  information  (Appendix  I). 


It  is  appreciated  that  there  are  many  practical  problem*  in  organiz¬ 
ing  and  administering  an  effort  of  this  kind.  Since  wounded  patients  are 
classified  as  surgical  patients  it  seems  logical  that  the  heads  of  the 
participating  surgical  divisions  and  their  supporting  residsnt  and  nursing 
staffs  should  be  sufficiently  interested  in  a  program  such  as  this, 
Modifications  in  the  program  of  data  collection  ar*  anticipated  once 
sufficient  feedback  information  is  developed. 

Hopefully,  as  mentioned  in  the  early  part  of  this  discussion,  useful 
relations  may  be  forthcoming  early  in  the  process  betwesn  missile 
parameters  and  subsystem  changes  and  between  subsyetsm  changes  and 
the  (biomechanical  output)  effectors,  Our  mathematical  and  statistical 
colleagues  tell  us  that  this  sffsctor-wound  relationship  is  a  stochastic 
one  for  two  reasons.  First,  there  is  a  distribution  of  wounds  within  ■ 
given  class  or  category  and  hsnce  there  is  a  random  variation  with  a 
certain  distribution  function  of  ths  corresponding  effector.  Second,  the 
effect  of  even  identical  wounds  sustained  by  different  individuals  also 
possasses  a  certain  distribution, 

There  ars  many  obvious  problems  involved  in  analyzing  the  data  to. 
be  collected.  Unfortunately  there  are  no  known  data  previously  collected 
from  which  normals  or  non-injured  standards  can  be  eetablilhed.  It 
is,  of  course,  impossible  to  obtain  pre-injury  effector  and  subsystem 
"normale"  for  the  hospital  cases.  Since  we  are  primarily  interested  in 
males  of  military  age,  the  choice  of  civilian  accidents  for  data  collection 
will  be  so  restricted.  However,  useful  information  may  be  gathered  for 
cases  in  which  injured  patients  would  return  following  complete  recovery 
Otherwise  independent  sample!  will  have  to  be  taken  from  normal 
personnel  In  order  to  obtain  pre-injury  distributions  of  the  appropriate 
subsyetsm  and  effector  parameters,  There  is  also  a  need  for  anthro¬ 
pomorphic  data.  Therefor*  they  are  included  under  special  studies 
(Appendix  1). 

Sources  of  Error  in  the  Procedures,  Th*  following  are  some  of  the 
more  apparent  errors  which  we  are  told  ihould  be  considered: 

1,  Errors  in  determining  the  group  normal  for  each  effector 
measure  inaemuch  a*  (a)  subjects  cannot  be  evaluated  before  wounding 
and  (b)  a  statistical  group  normal  must  b*  established  with  which  to  com* 
pare  the  individual  disability, 
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2.  Error  arising  from  individual  deviation  from  the  group  normal. 

3.  Error  arising  from  interaction  between  the  effectors.  These 
errors  are  minimized  by  attempting  to  isolate  the  effectors,  but  they  can¬ 
not  be  totally  eliminated. 

4.  Errors  arising  from  variation  in  the  measuring  and  aiseesing 
techniques  of  the  medical  and  technical  evaluators. 

5.  Errors  in  the  mechanical  devices  used  to  quantitate  the 
efficiencies  and  errors  associated  with  positioning  the  derives  on  the 
subjects , 


6.  Clinical  laboratory  error*. 

7.  Multi-clinic  errors  especially  where  mors  than  ons  hospital 
is  chossn  for  data  collsctlon, 

SUMMARY .  In  an  initial  effort  to  quantify  change*  in  human  perform¬ 
ance  due  to  mie tile  injury,  we  propose  to  collect  certain  clinical  informa¬ 
tion,  associated  missile  ballistic  factors,  and  measure  a  select  group  of 
neuro-museular  responses.  Such  responses  are  chosen  as  potential 
descriptors  for  man-machine  performance  problem!,  As  mentioned,  a 
man-machine  system  model  for  incapacitation  evaluation  is  being 
developed,  w  The  acquisition  of  detailed  human  clinical  data  is  essentially 
restricted  to  that  available  in  the  emergency  hospital.  Hopefully,  such 
information  will  permit  the  establishment  of  new  and  ussful  relations  early 
in  the  process, 
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I,  Anatomical  Actions 


Subscript  codes  for  anatomical  actions  of  the  upper  limb  are  given 
only  for  the  shoulder  region. 


Subscript  Code 

A0 

*1 

A, 


3 


8 

A9 

*10 

*11 

*12 

*13 

*14 

*15 

*16 

*17 

*18 

*19 

*20 


Anatomical  Actions 

Actions  of  the  Parts  other  than  upper  limb 

Rotation  of  scapula 

Adduction  of  scapula 

Raising  of  scapula 

Lowering  of  scapula 

Moving  of  scapula  forward 

Moving  of  shoulder  forward 

Lowering  of  shoulder 

Drawing  of  shoulder  backward 

Raise  shoulder 

Adduct  shoulder 

Flexion  of  arm 

Extension  of  arm 

Abduction  of  arm 

Adduction  of  arm 

Medial  rotation  of  arm 

Lateral  rotation  of  arm 

Flexion  of  forearm 

Extension  of  forearm 

Supination  of  hand  by  fortarm 

Pronation  of  hand  by  forearm 
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APPENDIX  A  (cont'd) 

II.  Muscles  of  the  Upper  Limb 

Subscript  codes  for  the  muscles  of  the  upper  limb  are  given  only  for 
the  shoulder  region. 

Subscript  Code  Name  of  Muscle 


Mi 

Deltoid 

m2 

Traperiu* 

M 

Suoclavius 

m4 

Pectoralis  major 

m5 

Sternocleidomastoid 

-6 

Sternohyoid 

M7 

Biceps 

M8 

Coraeobrachialis 

m9 

Pectoralis  minor 

M10 

Serratue  anterior 

-11 

Supraspinatus 

-12 

Levator  scapula 

-13 

Rhomboid  minor 

-14 

T  ricaps 

-15 

Infraspinatus 

M16 

Rhombcid  major 

Teres  minor 

Subscapularis 

M19 

Teres  major 

M20 

Latlssimus  dorsi 
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III.  Spinal  Nerves  of  the  Upper  Limb 

Subscript  codes  for  the  spinal  nerves  which  innervate  the  upper  limb 
muscles  are  given  in  association  with  the  shoulder  region  only, 


Subscript  Code 


Spinal  Nerve 


First  cervical  nerve  of  epinal  cord 
Second  cervical  nerve  of  spinal  cord 
Third  cervical  nerve  of  the  epinal  cord 
Fourth  cervical  nerve  of  the  epinal  cord 
Fifth  cervical  nerve  of  the  epinal  cord 
Sixth  cervical  nerve  of  the  spinal  cord 
Seventh  cervical  nerve  of  the  epinal  cord 
Eighth  cervical  nerve  of  the  epinal  cord 
Fir-'t  thoracic  nerve  of  the  epinal  cord 
Second  thoracic  nerve  of  the  epinal  cord 
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APPENDDC  B 

Wrisf  Dcrcriptioi'i  uf  luocrumentation  ior  Measuring  and  Recording 
Force -Time  Histories  for  Hand  Gra*ping  and  Flexion  and 
Extension  in  the  Wrist  and  Eibow 

The  device  ii  a  bipod-mounted  pistol  grip  (4-1/2  x  2  x  1  inch)  «pring 
loaded  with  three  internally  mounted  piezoelectric  crystals,  It  is  expected 
that  this  will  be  a  measure  of  grasping  ability  for  sudden  short  grasps 
and/or  prolonged  holding,  The  lead  zirconatetitinate  crystals  used  in  the 
prototype  model  have  been  dead-sveight  tested  to  500  pounds  with  no 
appreciable  non-linearity  in  electrical  response. 

Output  from  the  pressure  transducers  is  fed  into  a  battery  operated 
charge  amplifier  which  is  also  used  to  drive  a  2-channel  pen  recorder. 

The  amplifier  and  recorder  are  housed  in  a  portable  carrying  case  weigh¬ 
ing  approximately  10  pounds  and  with  an  overall  dimension  of  12"  x  8"  x  6", 

For  flexion  data,  removable  swivel  screws  are  attached  to  each  end 
of  the  hand  grip.  The  swivel  screws  rotate  in  a  U-clamp  which,  in  turn, 
ie  affixed  to  the  table  or  a  rigid  surface.  The  felt-backed  nylon  cords 
which  are  affixed  to  the  pistol  grip  are  grasped  in  the  palm  of  the  hand 
or  slipped  over  the  wrist.  Pulling  the  cord  orient*  the  meaiuring  device 
parallel  to  the  direction  of  the  flexion  force.  Removing  the  cord  and 
pushing  on  the  face  of  the  grip  will  produce  exteniion  measurements. 
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Behavioral  Information 

Activity  at  moment  of  wounding1. 

~  (check) 

Running  - - 

Standing  - 

Sitting  _ _ 

Lying  - - - 

Other  (explain)  _______  - - - - — - 


Remained  conscious: 

Yes  _____  No  — 

I£  no,  was  uncon*ciousne*s 
.Immediate  ?. 

Later  ?  ..  - •  - 

If  later,  about  how  long 


mins. 


Conoclou*  responses'. 

A,  Psychological: 
Highly  excited? 
Stayed  calm? 

B,  Physical: 
Started  fighting 
Ran 

Walked 
Stood  up 
Fell  to  ground 


Remarks 


1 

1 
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APPENDIX  E 

f*.  1  of  V>  i  nor  a  n  ri  7  vf*  rna1_  W  T  rarf  Tir»  £°  ®  ♦* 

Description  of  Clothing  Damage  on  Victim: 

Material  Description  of  Damage  (Hole  size,  etc.) 

Overcoat:  ____________  _ 

Jacket:  _ _  _ 

Shirt;  _ 

Undershirt: _ 

Pants:  _  _ _ 

Shorts:  _ 

Other: 


Wound  Extrance  ; 

Location(s)  of  Penetratlon( s)  Dimensions  of  Penetrations 

(a)  _ _ 

(b)  _ _ 

(c)  _  _ 


Wound  Exit: 

Location(B)  of  Exit  Hole(s)  Dimensions  of  Exit  Hole(s) 

(a)  _  _ 


1 


Clinical  Observations  (Coat'd) 
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Effector  Measurements 
(Upper  Limbs  Only) 

Admleslon 

Hand  Grasping; 

Sudden  maximal  effort:  ________ 

Prolonged  grasp  (loading  to  be  specified):  _ 

Grasping  follower  exercise  (to  be  specified):  •  •  ■  : 

Wrist  Flexion: 

Sudden  maximal  effort:  •  - 

Prolonged  flexion  Loading  to  be  specified): 

Wrist  Extension; 

Sudden  maximal  effort:  _____ 

Prolonged  flexion  (loading  to  be  specified): 

Elbow  Flexion: 

Sudden  maximal  effort:  _____ 

Prolonged  flexion  (loading  to  be  specified): 

Elbow  Extension: 

Sudden  maximal  effort:  ________ 

Prolonged  elbow  extension  (loading  to  be  _____ 

specified): 


+4hrs, 
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APPENDIX  H 
Information  on  Normals 

The  formats  anticipated  for  normals  are  duplicates  of  those  used 
in  Appendix  F  and  Appendix  G,  i.e.  ,  wherever  clinical  information 
is  sampled  there  is  an  assumed  need  for  normal  vtluea,  These  values 
will  either  be  drawn  from  similar  populations  and/or  from  those 

i  . 

victims  who  survive  the  injuries  and  are  considered  to  be  noJrrhftl 
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Age: 


Sex: 


APPENDIX  I 

Anthropometric  Information 
_  Weight: _ 


Physical  Measurements: 

Height;  ________ 

Weight: _ 


Head  circumference  (forehead  level): 

Vertical  distance  (top  of  head  to  bottom  of  mandible): 
Neck  length  (tip  of  hyoid  bone  to  eupraeternal  notch): 
Neck  circumference  (at  midpoint  of  neck  length): 
Cheit  circumference  (at  nipple  line): 

Abdomenal  circumference  (umbilical  level): 

Hip  level  circumference  (level  of  iliac  creet): 

Sternal  notch  to  aymphaiie  public; 

Spinoue  Procee*  of  C-7  to  coccyx: 

Width  of  ehouldere  (acromion  to  acromion): 


Right 


Acromion  to  radial  epicondyle: 

Midarm  circumference: 

Radial  epicondyle  to  radial  etyloid  proceee; 
Radial  etyloid  proceee  to  tip  of  middle  finger: 
Midforearm  circumference: 

Wriet  circumference: 

Anterior  iliac  epine  to  upper  patella; 

Upper  patella  to  eole: 


1  -4  -  ■■  ■  -  - 

I 
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Race: 
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(cont'd,  ) 

Right  Left 

Midcalf  circumference: 

Circumference  at  patella: 

Midleg  circumference: 

Ankle  circumference: 

Length  of  foot: 
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PROBLEMS  IN  THE  DESIGN  OF  STATISTICS 
-  GENERATING  WAR  GAME§ 

William  H.  Sutherland  ^ 

Research  Analysis  Corporation,  McLean,  Virginia 


A  newspaper  headline  last  week  said  "Helicopters  Crash  in  War  Games,  " 
Now  since  I  am  billed  as  presenting  a  problem  to  you  under  the  title  "Prob¬ 
lems  in  the  Design  of  Statistics  Generating  War  Games,  "  the  headline 
makes  me  hasten  to  tell  you  that  the  war  games  I'm  concerned  with  are 
not  at  all  like  the  newspaper  version,  and  that  whatever  statistics  that 
kind  of  war  games  generates  in  the  form  of  crashes,  such  statistics  have 
little  in  common  with  the  kind  I  wish  to  talk  to  you  about. 

The  kinds  of  war  games  that  are  of  concern  are  played  for  Army 
research  purposes.  They  are  two -sided,  somewhat  formal  exercises- 
flayed  indoors  using  maps  and  often  using  computers;  They  are  of  a 
size  and  complexity  measureabie  in  tens  of  man  months  of  play  and 
analysis  effort  (an  expensive  kind  of  effort  as  operation  research  studies 
go,  but  seldom,  1  suspect,  having  costs  comparable  With  the  kind  of  war 
games  in  the  newspaper  headline).  In  our  games  records  are  kept  of  the 
details  of  play,  and  from  these  records  statistics  are  derived.  The 
statistics  of  the  title,  then,  concern  not  real-world  helicopters  or  troops 
or  guns,  but  do  concern  the  helicopters  or  troop*  or  guns  which  the 
gamers  have  in  mind's  eye  as  they  play.  Battle  results  are  found  by 
applying  rules,  not  bullets,  and  sometimes  the  battle  results  are  made 
to  depend  on  random  numbers, 

The  players  usually  have  a  good  deal  of  freedom  of  action  tactically, 
and  it  is  in  this  sense  that  I  take  the  liberty  of  considering  war  games 
to  be  experiments,  (Certainly  the  games  do  have  this  in  common  with 
experiments:  we  never  know  how  they  are  going  to  come  out,) 

However,  they  are  unlike  most  experiments  in  that  one  would  not 
ordinarily  expect  to  be  able  to  repeat  the  initial  conditions  with  strict 
exactness.  If  one  were  to  try,  with  say  the  same  players,  they  would  no 
longer  be  in  the  dark  about  their  adversary,  because  of  what  they  had 
learned  in  the  first  game,  If  one  tried  a  second  set  of  players,  they 
would  necessarily  have  different  tactical  experience  inside  their  heads 
to  begin  the  game  with. 


s»<  - 


Design  of  Experiments 


68  6 

Now  after  telling  vou  that  the  tool- -war  gaming- -ie  one  that  in  one 
sense  defies  replication,  let  me  ask  you  about  a  problem  that  war  gamers 
face,  which  it  seems  to  me,  can  be  discussed  partly  in  terms  of  what 
replication  would  show  if  indeed  replication  were  possible.  The  problem 
presented  itself  in  the  course  of  a  Research  Analysis  Corporation  study 
on  the  use  of  war  gaming  as  a  research  tool.  It  is:  when  should  the  war 
game  designer  use  random  numbers  in  the  game ,  and  when  should  he 
avoid  using  them?  As  you  will  see,  the  guidelines  we  have  are  only 
qualitative  and  what  appear  to  us  to  be  common  sense,  It  would  help  if 
we,  had  better  guidelines,  and  this  is  the  problem  which  I  ask  the  panel 
to  consider,  So  much  for  the  introduction,  What  I  have  to  say  is  in 
three  sections;  (l)  the  reasons  for  using  random  numbers;  (2)  the 
appropriateness  of  using  average  values  versus  random  numbers;  and 
(3)  the  effects  of  random  numbers  on  interpreting  game  results. 

1.  REASONS  FOR  USING  RANDOM  NUMBERS.  Games  often  make 
use  of  random  numbers  as  a  way  of  deciding  details  of  combat,  The  use 
of  random  selections  from  previously  determined  or  estimated  probabilltlas 
serves  two  main  kinds  of  function;  to  represent  the  chance  nature  of  war¬ 
fare  and  to  keep  players  from  having  an  inappropriate  knowledge  of  their 
opponents, 

As  for  any  model,  the  chance  nature  of  warfare  can  of  course  be  only 
imperfectly  represented.  Only  a  few  of  the  most  important  chance  events 
can  be  selected  to  be  part  of  a  game.  I  suppose  that  thie  statement  will 
i.«m  a  little  like  Alice-in-Wonderland  to  this  audience  for  many  of  the 
papers  presented  in  this  conference  the  problem  seems  to  have  been  to 
nail  down  chance  events  which  wander  uninvited  into  the  problem  -  here 
we  are  in  effect  dragging  them  in  by  the  heels,  Of  these  chance  events 
the  use  of  random  selections  can  be  considered  as  a  means  of  at  least 
roughly  representing  the  consequences  of  groups  of  causal  sub-svsnts. 
These  sub-events  result  in  the  "main"  event  which  is  being  represented, 

The  sub-events  - -the  direct  causes--may  be  Impractical  or  impossible  to 
know,  or  they  may  in  the  game  be  unnecessary  to  know,  As  an  analogy, 
the  causes  for  the  small  deflections  of  a  bullet  fired  from  a  .gun  in  a  test 
stand  may  well  be  impracticable  to  know;  they  depend  on  such  matters 
as  changes  in  air  density  along  the  patch  of  successive  bullets  or'  slight 
and  unknown  asymmetries  in  the  loading  of  the  powder.  For  many 
practical  purposes,  it  is  not  necessary  to  know  the  causes.  So  also  in 
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war  games  a  spread  of  outcomes  for  an  event  can  be  used  without  specify¬ 
ing  the  causes.  Thus  random  numbers  can  be  used  in  games  to  represent 
events  whose  causes  are  not  thoroughly  understood  or  sometimes  to 
simplify  considerations  which  would  otherwise  be  in  too  great  a  detail 
considering  the  scale  of  the  game.  For  example,  in  a  recent  game  the 
outcome--succeus  o r failure- -of  a  minor  raid  against  a  logistics  installa¬ 
tion  behind  enemy  lines  was  represented  by  a  simple  draw.  Detailed 
consideration  of  the  complex  of  factors  for  and  against  the  raiders  was  not 
appropriate  considering  the  probably  minor  effects . of  the  raid  on  the 
overall  game  outcome. 

The  eacond  class  of  use  for  random  choice  from  distributions. is  to 
keep  the  opposing  players  suitably  in  the  dark  ae  to  the  exact  capabilities 
of  their  opponents,  and  thus  add  to  the  realism  of  the  game.  Without 
random  factors  a  player  might  be  able  to  work  the  model  or  formula 
backwards  and  find  enemy  strengths.  So  randomness  makes  the  players' 
decisions  more  like  what  they  would  be  in  real  life. 

Parenthetically,  it  has  been  observed  that  such  use  of  random  num¬ 
bers,  by  making  the  incidents  of  a  game  less  predictable,  make*  play 
more  interesting  to  the  participants,  This  contributes  to  the  intensity 
and  involvement  which  seem*  to  characterize  games,  and  which,  one 
hopes,  may  occasionally  result  in  an  otherwise  routine  tactic  being 
replaced  with  a  brilliant  one  which  may  alter  the  concept  being  studied, 

2.  APPROPRIATENESS  OF  AVERAGE  VALUES  VS  RANDOM  SELEC¬ 
TIONS.  Suppose  that  in  a  particular  theater  rain,  if  it  occurs,  has  a  strong 
effect  on  operations,  but  that  it  occurs  only  say  three  percent  of  the  ’■ime, 
and  without  any  repetive  pattern  (i,  e.  ,  in  a  way  that  is  reaspnably  repre¬ 
sented  as  random).  A  single  game  is  being  played,  and  the  random  number 
generator  indicates  that  it  is  to  rain  on  some  important  day  of  the  game. 

Is  it  appropriate  to  play  that  day  according  to  rainy  day  rules  ?  To  do  so 
would  make  the  play  for  that  day  "non-typical";  to  disregard  the  weather 
could  be  criticized  as  being  unrealistic,  So  one  subquestion  to  ask  about 
random  numbers  is- -when  is  it  appropriate,  for  matters  like  weather 
effects  to  use  "typical"  values  (i,  e.  ,  dry  weather);  when  to  use  average 
values  (i.  e.  ,  97%  dry),  ana  when  should  the  variations  be  randomly 
selected  ? 
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No  hard  and  fait  rules  come  to  mind  immediately,  but  certain 
observations  on  the  subject  can  be  made.  The  first  ie  elementary.  It 
is  that  there  i«  little  point  in  using  the  variations- -which  of  course 
complicate  the  game--unleis  the  factor  itself  is  important  to  the  game. 

If  in  the  example  rain  had  small  significance,  average  values  for  its 
effects  would  certainly  be  sufficient. 

Beyond  being  important  in  its  general  effect  on  the  game  output, 
though,  further  aspects  of  its  importance  need  to  be  examined,  One  is 
whether  the  variation  itself  has  an  impact.  If  the  effect  being  comidered 
does  mt  tend  to  do  what  I  call  'weight"  the  game  results,  then  there  may 
be  no  point  in  using  any  but  an  average  value  even  though  tha  overall 
effect  is  large, 

Let  me  explain.  One  concept  of  the  reason  for  playing  war  games, 
as  opposed  to  say  OR  analytic  studies  is  that  the  play  permits  examina¬ 
tion  of  certain  interactions  that  none  of  the  other  methods  seem  able  to 
examine.  Specific  aspects  of  the  interactions  between 

weapons, 
tactics,  and 
environment, 

can  be  studies  in  a  combined  manner  in  a  gome. 

Let  us  take  an  example,  As  part  of  a  recent  war  gams  at  Research 
Analysis  Corporation  two  competing  antiaircraft  systems  wsrs  compared 
in  the  role  of  defending  against  armed  helicopters,  The  particular 
tactics  used  by  the  helicopters  included  hiding  behind  available  ridges 
and  then  coming'  in  fast.  The  particular  hilly  terrain  of  the  gams 
limited  the  range  at  which  the  helicopters  could  be  acquired  as  targets, 
For  these  tactics,  then,  and  this  terrain  the  more  sophisticated  and 
expensive  weapon  was  not  usable  at  the  long  ranges  at  which  it  was 
effective.  The  less  complex  weapon  did  nearly  as  well.  The  statistic 
used  for  the  comparison  was  simply  the  number  of  helicopter*  shot 
down  per  helicopter  sortie.  In  general,  then,  the  game  may  be  looked 
at  as  a  method  of  examining  the  interactions  between  the  weapons  and 
the  terrain,  and  the  weapons  and  the  tactics,  or  for  that  matter,  any  of 
the  three  as  affected  by  the  other  two,  Quantitative  weights  are  implicit 
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ii>  iuc  >jiay  ui  uic  jjame  ana  can  be  thought  oi  as  contriDuting  to  tne  relative 
use  and  relative  effectiveness  of  the  weapons  or  tactics  under  the  game 
conditions.  The  game  provides  information  and  insights  concerning  such 
interactions,  on  the  basis  of  such  implicit  weighting.  The  combining  of 
tactics,  weapons  capabilities,  and  surrounding,  may  thus  be  thought  of 
as  being  a  product  or  output  beyond  the  previous  knowledge  of  the  individ¬ 
ual  inputs.  These  are  the  experimental  results  with  which  we  work,  The 
game,  in  my  viewpoint,  is  a  particularly  suitable  and  unique  tool  for 
Army  use,  partly  because  it  can  study  such  interactions. 

To  return  to  random  numbers,  there  is,  I  suggest,  little  to  be  gained 

in  using  them  to  produce  variability  in  some  computational  input  or  output 
unless  such  a  range  of  numbers  is  tied  to  chis  weighting,  The  nature  of 
the  tis-in  is,  as  far  as  1  know,  an  unexplored  area  and  one  to  which  you 
may  be  able  to  suggest  approaches. 

3.  INTERPRETATION  OF  GAME  RESULTS  -  NUMBER  OF  RANDOM 
NUMBERS.  But  In  order  to  do  so  with  better  imight,  we  might  also  look 
at  the  effects  of  using  random  numbers  on  the  interpretation  oi  game 
results.  As  I  indicated,  any  single  play  would  have  to  begin  a  little  . 
differently  from  any  other  play,  and  would  then  continue  differently,  even 
though  the  general  circumstances  of  the  game  were  similar,  This  comes 
about  because  the  decisions  of  (all  too)  human  players  are  involved,  but 
also  because  of  random  number  use.  While  the  overall  variability  in 
results,  which  makes  interpretation  difficult,  cannot  really  be  separated 
into  the  two  causes,  we  can  for  the  time  being  assume  that  the  players 
could  be  given  only  limited  choices,  and  we  can  concentrate  on  the  vari¬ 
ability  caused  by  random  numbers.  In  effect  this  is  what  does  happen  in 
some  computer  simulations.  Consider  the  number  of  times  a  random 
number  is  chosen  for  a  particular  purpose  in  the  course  of  a  gams.  In 
practice  this  varies  greatly  from  one  game  to  another.  Some  use  random 
selections  literally  thousands  of  times  in  one  play:  others  have  few, 
and  one  two-sided  exercise  that  was  called  a  game  did  not  use  any,  Let 
us  examine  a  little  the  consequences  of  the  use  of  different  numbers  of 
random  selections,  If  the  purpose,  for  example,  was  to  find  the  battle 
outcome  for  a  single  highly  important  battle,  and  the  random  choice  was 
made  just  once,  then  two  quite  different  possible  results  could  happen. 

In  statistical  terms,  the  spread  of  potential  results  would  be  large. 
(Incidentally  what  would  be  learned  from  such  a  game  would  be  little.  ) 

On  the  other  hand,  if  a  long  series  of  battles  were  fought,  each  of  which 
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was  equally  important,  and  in  which  the  probability  of  the  different 
possible  result*  did  not  change  b<*tw»  mmwacbi  men  •tatistically  the 
relative  spread  of  possible  overall  results  would  be  small.  Of  course 
war  games  really  do  not  present  a  picture  of  a  large  number  of  exactly 
similar  events  decided  on  a  probability  basis  in  the  way  described. 

Still,  they  presumably  behave  somewhat  as  if  they  did,  and  thus  to  a 
limited  extent  the  spread  of  results  can  be  discussed  as  a  statistical 
matter.  The  limitations  include  this:  that  the  measure  of  outcome 
is  indeed  a  statistic-- "winning"  or  "losing"  is  probably  not  such  a  mea¬ 
sure,  Secondly,  for  some  mathematical  considerations  the  measure 
must  be  an  average.  Certainly  not  all  the  statistics  with  which  we  are 
concerned  are  averages.  But,  to  the  degree  that  the  measure  of  outcome 
used  is  a  statistical  average,  the  greater  the  number  of  random  events 
that  are  applied  directly  to  this  outcome,  the  smaller  the  standard  devia¬ 
tion  of  the  result.  The  hypothetical  universe  of  means  we  are  speaking 
of  gets  relatively  narrow  as  more  choices  are  put  in.  Its  standard 
deviation  would  vary  inversely  with  the  square  root  of  the  number  of 
random  choices  made.  Thus  certain  aspect*  of  thole  games  which  repeat¬ 
edly  make  a  very  large  number  of  draws  may  be  thought  of  as  giving  the 
same  results  as  if  average  values  had  been  used  in  the  computations. 

A  complication  involved  in  warlike  simulations  is  that  the  entities 
one  deals  with  (unit*  or  weapons)  maybe  destroyed  or  eliminated,  Thu* 
one  has  a  decreasing  set  of  pieces  to  work  with,  and  the  relationship 
between  the  opposing  pieces  can  well  be  changing  as  the  work  progresses. 
Thus,  if  a  statistic  is  used  like  'the  average  number  of  Red  entities 
destroyed  per  Blue  counter-entity,  "  such  an  average  could  be  quite  differ¬ 
ent  at  the  beginning  of  a  game  than  toward  the  end,  when  perhaps  each 
side  would  at  least  need  to  do  more  hunting  to  find  the  enemy. 

What  ehould  a  game  designer  do  then?  Should  he  try  to  cut  down 
the  quantity  of  random  numbers  hie  game  ie  to  u»e,  or  try  to  increaee 
them  in  the  hope  of  reducing  the  deviation  of  possible  results?  No  general 
rule  seems  evident  at  this  tirpe, 


THE  FUTURE  OF  PROCESSES  OF  DATA  ANALYSIS* 

John  W.  Tukey 

Princeton  University  and  Bell  Telephone  Laboratories 

I  am  here  to  speak  of  the  near-middle  future  of  the  processes  of  data 
analysis.  This  can  only  be  done  by  indirection,  since  any  processes  that 
serve  as  examples  must  be  drawn  from  the  present  of  near  future,  but 
we  can- use  such  examples  to  illustrate  what  may  be  hoped  to  be  broadly- 
applicable  principles  of  continuing  importance. 

In  general,  the  future  of  processes  of  data  analysis  is  rosy,  but  it 
is  not  yet  clear  how  fast  the  sun  is  rising.  The  modern  computer  has 
offered  us  many  opportunities  --  far  more  than  we  have  seized  --  and 
there  have  been  many  more  opportunities  for  innovation  that  do  not  require 
a  computer  than  we  have  seized.  Looking  at  the  last  decade  or  two,  it 
is  clear  that  we  have  made  much  progress  --  but  we  cannot  be  content 
with  the  rate  at  which  we  have  gone.  Will  we  do  enough  better  in  the 
future?  Will  we  try  to  find  approximate  (or  even  crude)  answers  to  more 
pressing  problems,  or  exact  answers  to  problems  of  limited  (or  non¬ 
existent)  relevance?  Who  can  say?  (For  a  more  historical  and  less 
specific  discussion,  see  Tukey  1965.  ) 

1.  SOME  PRINCIPLES.  To  point  toward  the  near -middle  future, 
we  begin  by  stating  a  number  of  broad  principles  concerning  the  pro¬ 
cesses  of  data  analysis  (a  phrase  that  ought  to  be  construed  as  including 
the  thoughts  of  the  analyst  of  data  as  well  as  his  manipulations)  which 
we  expect  to  retain  their  importance: 

Two  major  aspects  of  such  processes  will  continue  their  great 
importance: 

(1)  the  essential  erector-set  character  of  data-analysis  techniques, 
where  any  2,  3  or  4  techniques  are  likely  to  be  combined  without  warn¬ 
ing, 


(2)  the  steadily  decreasing  cost  (and  a  so-far  only  slowly  increas¬ 
ing  ease)  of  computation,  which  is  reflected  in  an  ever-increasing 


*Prepared  in  part  in  connection  with  research  at  Princeton  Univer¬ 
sity  sponsored  by  the  Army  Research  Office  (Durham). 
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emphasis  on  computer  usage  and  an  ever-growing  role  of  computer-unique 
contributions  and  processes. 

Disproportionately  rapid  expansion  will  continue  to  repair  past 
deficiencies  ini 

(3)  graphically  and  informality  of  processes  of  analysis, 

(4)  graphicality  and  incisiveness 

(5)  flexibility  and  fluidity 

(6)  empirical  discovery  of  techniques 

(7)  focusing  and  parsimony, 

In  support  of  these  Improvements,  our  conceptual  frameworks 
will  give  more  and  more  attention  to; 

(8)  doing  the  approximately  right,  rather  than  the  exactly  wrong 
(including  dropping  tight  specifications  as  rapidly. and  generally  as  we 
may). 

(9)  using  umbra-penumbra  modal  pairs  and  other  simultaneous 
(rather  than  alternative)  modal  combinations. 

(10)  making  the  relation  of  estimator  and  target  a  two-way  street, 

And  the  day  will  yet  dawn  when; 

(11)  there  will  be  one  or  more  programming  system!  appropriate 
to  data  analysis, 

As  they  stand,  thsse  principles  are  mainly  unexplained  worde, 
requiring  both  example*  and  discussion  to  makt  then  more  understand¬ 
able, 


2,  A  GROUP  OF  EXAMPLES.  Let  us  turn,  then,  to  a  group  of 
examples,  to  instances  of  specific  areas  where  progress  is  current.  (I 
am  sure  my  selection  ie  biased,  but  this  is  only  to  be  expected,  )  These 
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examples  have  not  been  selected  to  match  the  principles  in  a  one-to-one 
way,  instead  each  has  been  chosen  to  illustrate  a  few  principles,  with 
an  attempt  to  illustrate  each  principle  more  than  once,  (Unfortunately, 
principle  11  cannot  be  adequately  illustrated, ) 

Table  1  lists  the  examples  and  indicates  their  closest  relations  to 
specific  principles. 


I 

THE  EXAMPLES 

3.  NEWER  APPROACHES  TO  TYPICAL  VALUES,  It  has  long  been 
recognised  that  samples  from  distributions  whose  tails  straggle  more 
than  those  of  a  Gaussian  were  not  well  summarised  by  an  equally  weighted 
arithmetic  mean,  The  procedures  suggested  by  Jeffreys  (1938,  see  also 
Newcomb  1886)  for  large  samples  have  been  occasionally  implemented 
(e.g,  Hulme  end  Symme  1939).  More  recently  (i.  e.  ,  Tuksy  I960,  Hodges 
and  Lehmann  19 62 ,  Tukey  and  McLaughlin  1963)  it  has  been  recognised 
that  what  is  nseded  is  not  so  much  a  large- sample  technique  carefully 
bent  to  fit  the  particular  distribution  at  hand,  but  rather  techniques  which 
provide  relatively  high  efficiency  over  a  wide  range  of  distributions  -- 
techniques  that  are  (approximately)  robustly  efficient  as  well  as  being 
(approximately)  robustly  valid, 

Much  is  being  dona  in  this  area,  and  we  ehall  soon  have  not  only  a 
body  of  asymptotic  theory  (Lehmann  1963a,  b,  c,  1964,  Huber  1964, 

Bucksl  1964)  but  an  array  of  directly  useful  techniques  (Hodges  and 
Lehmann  1962,  1963,  H^yland  1964,  Dixon  and  Tukey  196?), 

The  problem  of  typical  values  in  the  plane,  and  in  higher  dimensions, 
is  not  so  simple,  since  there  is  no  obvious  affine -invariant  generalisa¬ 
tion  of  the  notions  of  order  statistics,  which  have  played  s  central  role 
in  the  one-dimensional  case,  Gentleman  (196?)  is  tackling  this  problem 
from  the  point  of  view  of  minimizing  p-th  power  deviations,  Elashoff 
and  Bickel  (1964")  are  investigating  Winsorizing  and  trimming.  Soon 
we  may  expect  working  tools  for  this  case,  too, 

Extensions  to  many  other  problems  are  obviously  needed,  and  can 
be  expected  to  occupy  both  asymptotic  theorists  and  practical-technique 
designers  over  a  considerable  period, 
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4.  NEWER  DISSECTIONS  OF  FACTORIAL  TABLES,  For  a  long  time 
only  two  dissections  of  data  arranged  in  a  two-  (or  more-)  way  table  were 
common  in  data  analysis.  Both  of  these  were  almost  always  left  implicit 
rather  than  made  explicit.  I  refer,  of  course,  to  the  additive  decomposi¬ 
tion,  whose  two-way  form  is 


y. .  =  y 
ij 


+  (yi-  'y.  . )  +  (yij"y.  . )  +  (yij “yi.  “y,  j+y,  . ) 


that  underlies  the  analysis  of  variance  for  crossed  and  nested  factors, 
and  the  multiplicative  decomposition,  whose  two-way  form  is 


n. .  =  n 

ij  ++ 


that  underlies  the  chi-square  test  for  independence  in  contingency  tables. 

The  cases  where  the  labels  of  the  columns,  or  the  labels  of  the  rows, 
or  both,  are  at  least  ordered  (and  perhaps  even  relevantly  quantitative) 
are  important  and  deserve  much  attention.  They  are  not,  however,  part 
of  the  subject  we  wish  to  discuss  here.  Our  immediate  concern  is  with 
decompositions  other  than  the  usual  ones  which  can  be  carried  out  on 
any  two-way  (or  more -way)  table. 

Among  the  earliest  of  these  was  the  separation  of  one  degree  of 
freedom  for  non -additivity  (Tukey  1949)  in  which  the  "row"  and  "column" 
parts  of  the  usual  decomposition  were  used  to  identify  and  separate  part 
of  the  "interaction"  parts.  Further  discussion  (Tukey  1955,  Scheffe  1959, 
Elston  1961),  sorr°  generalizations  (Ward  a-/’  Di~k  19 52)  and  various 
modifications  r>t  .  <  .  v  iarter  an4  T,  .  ,  L  -\->l  1959,  Mandel 

1961)  followed. 

The  apparent  needs  of  specific  data  analysis  produced  an  extension 
along  the  lines  of  the  "vacuum  cleaner"  (Tukey  1962)  which  does  not 
function  well  in  practice  without  the  aid  of  some  preliminary  prepara¬ 
tion  (e.  g.  FUNOR-FUNOM,  see  Tukey  1962).  This  is  only  one  of  a 
branching  family  of  alternatives  that  are  still  unexplored. 
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Some  directions  in  which  we  ought  to  go  are  clear,  but  the  details  of 
tools  and  formulations  are  far  from  settled,  We  need  to  dissect  a  two- 
way  table  in  more  parts  than  the  four  indfrat-»H  above.  It  will  soI41aiime» 
suffice  to  have: 


(al)  An  over-ail  contribution. 

(a2)  Column  contributions . 

(a3)  Row  contributions. 

(a4)  Unusual  cell  contributions, 

(a5)  Routine  cell  contributions. 

As  well  as  being  important  on  their  own,  such  dissections  clearly  have 
a  close  relation  to  tho  problems  of  Section  3, 

Except  for  the  smallest  tables,  it  is  likely  to  be  necessary  to  go 
further,  dissection  row  and  column  behavior  into  the  unusual  and  the 
routine,  just  as  for  cell  contributions.  In  either  case,  ws  will  be  pre¬ 
pared  for  both  of  these  extremea; 

(b)  row  and  column  effects  clearly  visible  above  a  "noise"  of 
routine  cell  contributions, 

(c)  a  few  cells  deviating  widely  from  all  the  others,  which  show  no 
pattern  of  variation  (including  none  by  row  or  column), 

We  will  be  prepared  for  oither  extreme,  since  we  shall  be  prepared 
for  any  mixture  of  these  extremes, 

We  are  here  at  a  very  early  stage  in  the  gaining  of  understanding, 
We  have  had  some  experience  in  the  identification  of  unuaualness,  but 
we  undoubtedly  have  much  to  learn.  Once  we  are  in  reasonable  shape 
for  two-way  tables,  there  are  many  waye  to  go, 

5,  SPECTRUM-LIKE  TECHNIQUES,  The  application  of  Fourie r 
methods" to  data  gave  rise  to  user.l  results  in  ths  simplest  cases  (e,  g. 
Whittaker  and  Robinson  1924,  Bartels  1940),  The  modern  era  in  this 
area  begins  with  the  recognition  (Bartlett  1950,  Tukey  1950)  that 
"white  noise"  is  almost  always  a  foolish  null  hypothesis,  and  that 
"white  noise  plue  a  few  sharp  lines"  was  an  equally  poor  alternative 
hypothesis.  Attention  was  first  directed  toward  such  questions  a* 
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consistent  estimation  of  spectrum  density  (which  the  writer  find.  miite 
uninter. ?tir.g,  sim-e  he  never  saw  even  an  approximately  infinite  amount 
of  time  series  data)  and  variability  (under  Gauesian  assumptions)  of 
estimates  of  averaged  spectrum  densities.  Later  developments  have 
emphasized  the  importance  of  keeping  close  touch  with  the  average  value 
of  one's  spectrum  estimates  and  the  advisability  of  introducing  a  variety 
of  new  techniques  in  order  to  approach  the  specific  problems  that  are 
important  in  the  specific  application.  (See  Technometrics  1961  for  a 
general  introduction,  including  complex  demodulation,  see  Akaike  1962 
for  misbehavior  of  the  aucocovariance  function,  see  Akaike  and  Yamamouchi 
1962  for  practical  problems  in  the  use  of  cross-spectra,  see  Hasselman, 
Munk  and  MacDonald  1963  for  the  bispectrum,  see  Bogert,  Heal y  and 
Tukey  1963  for  the  cepstrum,  cross-cepstrum,  pseudoautocovariance, 
and  related  concepts,  etc.  ,  see  MacDonald  and  Ward  1964  for  interest¬ 
ing  prediction- studying  techniques,) 

The  two  beliefs,  both  quite  erroneous  in  the  writer's  view,  that  have 
contributed  the  most  to  delays  and  inadequancies  in  the  use  of  spectrum 
analysis  have  been: 

(a)  A  belief  that,  in  using  spsctra,  one  ought  to  be  concerned  only 
with  Gaussian  situations, 

(b)  A  belief  that,  in  uaing  spectra,  one  ought  to  be  concerned  only 
with  stationary  situations. 

It  is  true  that  average  value  and  spectrum  only  complete  the  speci¬ 
fication  of  an  eneemble  of  time  series  if  ws  know  mors,  say  that  the 
ensemble  ie  Gaussian,  This  is,  however,  no  more  than  the  analog  of 
the  (equally  correct)  statement  that  average  value  and  variance  only 
complete  the  specification  of  a  distribution  when  we  know  more,  say 
that  the  distribution  is  Gaussian,  (Ws  do  not,  howaver,  confine  our  use 
of  variance  to  Gaussian  distributions.) 

In  the  absence  of  a  aultably  mathematical  formulation  and  treatment 
of  spectra  for  nonstationary  ensembles,  there  has  been  an  unfortunate 
tendency  for  aome  workers  to  feel  that  spectrum  techniques  should  only 
be  applied  to  situations  of  apparent  atationarity.  In  practice,  this  can 
be  quite  foolish,  as  Munk  and  Snodgrase's  discovery  (1957)  through  their 
nonetationarity,  of  the  weak  long-period  ocean  waves  arriving  on  our 
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Pacific  Coast  from  the  Indian  Ocean  and  beyond,  illustrates.  In  theory, 
it  is  at  best  dubious,  since  if  our  universe  should  repea>  itself  every 
11  12 

10  to  10  years,  the  whole  universe  (with  ail  its  time  scries)  may  per¬ 
haps  be  thought  of  as  stationary  --  and  who  can  deny  such  a  possibility? 

It  may  prove  fortunate  that  a  mathematical  formulation  of  the  non¬ 
stationary  case  is  now  at  hand  (Priestley  1965)  which  tells  us  to  do  for 
slowly  changing  spectra  just  what  we  have  done  for  plausibly  constant 
spectra. 

In  addition  to  the  new  types  of  quantities  being  introduced  and  used, 
we  are  in  the  middle  of  a  change  in  the  actual  computing  techniques  used 
to  process  the  data.  Where  once  some  subsequence  of: 

(cl)  taper 
(c2)  prewhiten 

(c3)  form  mean  lagged  products 
(c4)  apply  lag  window 
(c5)  Fourier  transform 
(c6)  hann  or  hamm,  etc. 

was  relatively  standard,  alternative  approaches,  involving  more  computa¬ 
tions  linear  in  the  observations  before  the  formation  of  squares  of  products 
involving  the  data,  are  in  use  or  contemplation.  Techniques  using  com¬ 
plex  demodulation  appear  to  involve  very  real  advantages,  and  are  already 
in  routinf*  use  (M.  D.  Godfrey  1964*).  Now  that  complete  Fourier  trans¬ 
formation  for  N  o*.'  vrf’ons  requires  only  a  few  times  N-  log  N  multi- 

plications  rather  than  N”  (Cooley  and  Tukey  1965),  we  may  well  see 
computational  techniques  develop  which  start  by  complete  Fourier  trans¬ 
formation  of  the  entire  data.  (The  spectrum- analytic  character  of  these 
techniques  will  be  revealed  by  wtot  happens  next  to  the  Fourier  coefficients 
and  how  the  ultimate  quantities  are  interpreted.) 

In  economics,  spectrum  analysis  is  currently  being  applied  to  the 
problem  of  seasonal  adjustment,  and  as  a  consequence  economists  are 
again  thinking  about  the  difficult  question  of  what  seasonal  adjustment  is 
really  supposed  to  do. 

So  far  as  one  can  now  see,  spectrum-like  analysis  is  going  to  continue 
to  ramify  and  develop  at  a  substantial  rate. 
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6.  UNRESTRAINED  MONOTONE  TRANSFORMATION,  The  fight  bet¬ 
ween  those  who  feared  the  loss  of  knowledge  that  comes  from  analyzing 
unwisely  expressed  data  and  those  who  feared  serious  biasing  of  levels  of 
significance  and  confidence  would  come  from  expressing  the  data  in  the 
way  in  which  it  seemed  to  like  to  be  expressed  is  an  old  one,  but  one 

that  has  never  reached  the  front  pages,  Partly  this  has  been  because 
changes  in  modes  of  expression  have  seemed  unimportant.  Partly,  I 
fear,  it  has  been  because  those  who  realized  that,  in  practice,  100%  and 
200%  improvements  in  efficiency  come  more  frequently  from  such  changes 
than  from  almost  anything  else  the  analyst  of  data  can  do  once  the  data 
is  taken,  have  not adve rtised  this  fact  sufficiently, 

Those  who  have  sought  better  modes  of  expression  have  traditionally 
chosen  some  simple  family  of  transformations,  often  i  =  (y+c)P,  and 
have  tried  to  choose  the  few  parametric  constants  wisely  in  each  particu-  ' 
lar  instance.  (For  a  clear  exposition  of  a  highly  developed  form  of  this 
approach,  see  Box  and  Cox  1964,.)  As  the  techniques  have  become  more 
explicit,  the  hope  of  their  wider  application  has  increased  steadily. 

All  this  continues  to  be  important,  but  the  pressure  of  a  real  need 
for  better  multidimensional  scaling  has  brought  about  a  computer-aided 
revolution.  The  work  of  Shepard  (1962,  1963)  and  Kruskal  (1964a,  b)  has 
shown  how  much  can  often  be  gained  by  letting  the  computer  choose  wb  - 
ever  monotone  transformation  of  the  original  value  will  lead  to  the 
aimplest  analysis,  The  impact  upon  multidimensional  scaling  and 
factor  analysis  is  already  substantial.  Kruskal1  s  reanalyses  (196?)  of 
Box  and  Cox  examples  show  that  even  a  3x3x3  experiment  may  be  big 
enough  for  such  an  analysis  to  be  fruitful.  We  can  hope  for  similar 
progress  in  many  other  areas  (although  semi-classical  results  on 
"Maximalkor relation"  show  that  we  cannot  do  it  everywhere). 

7.  ORDERED  PLOTS,  The  classical  example  of  plotting  observed 
values  rearranged  in  increasing  order  is  the  use  of  "probability  paper" 
to  show  the  apparent  Gaussianity,  or  absence  thereof,  of  a  sample  of 
observations,  This  example  is  classical,  but  it  is  still  surprising  how 
many  statisticians  have  had  little  contact  with  the  technique, 

The  arrival  of  the  half  normal  plot  (Daniel  1959)  introduced  a  major 
change  into  the  analysis  of  unreplicated  and  fractionated  2P  experiments. 
The  idea  that  a  set  of  contrasts  could  be  used  to  show  forth  the  unusual 
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size  of  its  largest  values,  if  any  of  their  sizes  were  truly  unusual,  is  not 
a  difficult  one.  It  is  perhaps  surprising  that  it  took  so  long  to  appear. 

Later,  the  more  general  technique  of  "gamma  plotting",  in  which  two 
parameters  require  estimation  rather  than  one,  was  developed  and 
applied  in  a  variety  of  directions  (Wilk  and  Gnanade  sikanl96l,  1964a, 

Wilk,  Gnanadesikan  and  Huyett  1962). 

Today,  the  problem  of  adapting  these  techniques  to  the  general 
analysis -of-variance  situation,  where  different  mean- squares  have 
different  numbers  of  degrees  of  freedom  is  being  actively  attacked  with 
interesting  results  (Wilk  and  Gnanadesikan  1964b,  Wilk,  Gnanadesikan  and 
Lauh  1964). 

As  a  consequence  of  this,  the  writer  is  convinced  that  we  shall  see  a 
partial  return  of  the  pendulum,  which  has  now  swung  from  analyses 
guided  only  by  the  natural  order  of  lines  (and  the  relations  between 
average  mean  squares)  of  the  analysis  of  variance  to  analyses  guided 
only  by  the  relative  sizes  of  the  mean  squares.  I,  for  one,  believe  that 
we  ought  to  expect  attention  to  both  considerations  in  well  thought -through 
analyses,  though  in  ratios  differing  widely  from  instance  to  instance. 
(Given  a  complete  2^,  for  instance,  whose  409S  contrasts  behave  exactly 
like  a  Gaussian  sample,  1  would  regard  the  fact  that  the  12  largest  con¬ 
trasts  were  the  12  main  effects  as  nonaccidential  and  highly  significant.  ) 

Here,  too,  we  can,  1  believe,  lift  the  curtain  of  the  future  a  little 
When  I  try,  I  see  signs  of  plots  of  gaps  (=  spacings)  among  the  ordered 
observations  appearing  alongside  --  and  even  in  partial  replacement  of-- 
the  more  classical  plots  of  the  raw  ordered  observations.  Time  will 
tell. 


8.  HANGING  RQOTOGRAMS.  This  example  is  included  to  show 
that  even  among  the  simplest  of  graphical  techniques  there  can  be  new 
and  useful  techniques. 

The  histogram,  with  its  columns  of  area  proportional  to  number, 
like  the  bar  graph,  is  one  of  the  most  classical  of  statistical  graphs. 

Its  combination  with  a  fitted  bell- shaped  curve  has  been  common  since 
the  days  when  the  Gaussian  curve  entered  statistics.  Yet  as  a  graphical 
technique  it  really  performs  quite  poorly.  Who  is  there  among  us  who 
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can  look  at  a  histogram-fitted  Gaussian  combination  and  tell,  reliably, 
whether  the  fit  is  excellent,  neutral,  or  poor?  Who  can  tell  when  the 
fit  is  poor,  of  what  the  poorness  consists?  Yet  these  are  just  the  sort 
of  questions  that  a  good  graphical  technique  should  answer  at  least 
approximately. 

How  can  we  do  better?  If  we  have  observed  n.  cases  in  the  ith 

1 

class,  we  know  that  the  variance  of  n.  is  reasonably  proportional  to  its 
average  valves  (at  least  so  long  as  n^  is  not  a  large  fraction  of  the  total 
number  of  cases,  n+). 

If  we  are  to  do  a  reasonable  job  of  assessing  fit,  we  deserve  to  have 
roughly  constant  variance.  We  can  do  this  by  replacing  n.  by  VnT,  as 

we  are  well  aware  in  other  contexts.  We  can  do  the  same  here,  at 
least  for  the  case  of  classes  of  equal  width.  We  have  only  to  take  the 
square  root  (of  the  height)  of  the  fitted  curve  at  the  same  time  that  we 
take  the  square  roots  of  the  counts. 

Because  of  the  simple  identity: 

Vone  Gaussian  density  =  (constant)  (another  Gaussian  density) 

the  picture  will  look  much  the  same  --in  the  large  --a  family  of 
rectangles  compared  with  a  Gaussian  curve,  but  now  variability  is 
nearly  constant  (at  the  price  of  giving  up  the  principle  of  "equal  area 
for  equal  count"  which  has  real  uses  in  other  directions  but  few  if  any 
in  connection  with  goodness  of  fit). 

But  we  are  still  comparing  the  ends  of  a  row  of  rectangles  with  a 
curve,  something  the  human-eye -and-brain  combination  is  less  than 
perfect  at.  How  do  we  improve  matters  here?  We  have  only  to  say, 
carefully  and  precisely,  what  we  have  always  done,  in  order  to  learn 
what  we  might  better  do.  Classically,  we  have  taken  a  stack  of 
rectangles,  fixed  one  end  of  each  on  a  horizontal  line  and  compared 
the  other  ends  with  a  curve.  It  is  not  a  great  step  to  say:  "Let  us 
take  our  stack  of  rectangles,  fix  one  end  of  each  on  a  curve  and  com¬ 
pare  the  other  ends  with  the  straight  line .  Why  did  we  not  do  it  long 
ago?"  ~™ 
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While  we  are  about  it,  we  might  as  well  turn  the  picture  over,  letting 
the  curve  hang  down,  supporting  the  rectangles.  This  third  change  com¬ 
pletes  our  path  to  the  "suspended  rootogram"  in  which  the  eye  can  do  so 
much  more  for  us.  (Some  viewers  prefer  to  stop  at  the  "hanging  rooto¬ 
gram"  stage.)  Figures  1,2,  3,4  [at  the  end  of  this  article]  show  succes¬ 
sive  stages  in  the  progress  from  conventional  histogram  to  hanging 
rootogram. 

There  are  other  simple  things  to  do  in  the  graphical  area,  as  we 
shall  learn  as  we  take  care  to  realize  that  graphs  can  and  should,  among 
other  things,  be  used  for  diagnosis  as  well  as  naive  exhibition. 

9.  DEOMNIBUSING.  The  first  step  in  data  analysis  is  often  an 
omnibus  step.  We  dare  not  expect  otherwise,  but  we  equally  dare  not 
forget  that  this  step,  and  that  step,  and  other  step,  are  all  omnibus  steps 
and  that  we  owe  the  users  of  such  techniques  a  deep  and  important  obliga¬ 
tion  to  develop  ways,  often  varied  and  competitive,  of  replacing  omnibus 
procedures  by  ones  that  are  more  sharply  focused. 

The  replacement  of  group  comparisons  by  multiple  comparisons  has 
been  one  of  the  outstanding  phenomena  of  the  last  decade  and  a  half.  It 
has  raised  many  deep  issues  on  which  we  are  far  from  being  completely 
agreed  --  whose  discussion  would  take  more  space  than  we  can  here 
provide.  So  we  note  here  only  that  a  full  account  of  the  short-cut  methods 
using  ranges  both  in  numerator  and  denominators  is  at  last  appearing 
(Kurtz,  Link,  Tukey  and  Wallace  1965,  196?). 

We  note  also  that  progress  has  also  been  made  on  the  deomnibusing 
of  contingency  table  chi-square. 

The  detection  of  differences  in  the  effects  of  ordered  treatments  -- 
under  circumstances  where  the  effects,  if  any,  may  be  expected  to  be 
directly  --  or  antithetically  --  ordered  has  at  last  engaged  the  attention 
of  technique  manufacturers.  Two  competing  approaches  exist,  about 
which  all  protagonists  will  agree  that  either  one  is  to  be  preferred  to 
the  unwise  use  of  a  flabby  group  comparison.  One  procedure  is  developed 
in  a  framework  of  successive  testing  (Bartholomew  1959,  1961a,  b);  the 
other  in  a  framework  of  single  contrasts  of  maximizing  the  least  sensi¬ 
tivity  (Abelson  and  Tukey  1959,  1963).  (The  writer  notes  a  continuing 
preference  for  the  latter,  based  on  what  he  regards  as  good  reason. 

Again  space  bars  further  discussion.  ) 
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Still  more  recently,  there  is  progress  in  the  deomnibueing  of  "good¬ 
ness  of  fit"  tests,  which  have  always  had  so  omnibus  a  character.  For 
small  samples,  or  compulsory  heavy  grouping,  we  need  not  merely  sum 
the  squares  of  standardized  (or,  better,  Studentized)  deviations  to  find  a 
chi  square.  As  has  long  been  known  (e.  g.  Cochran  1954)  we  can  introduce 
any  convenient  set  of  orthogonal  comparisons,  and  evaluate  the  result* 
as  separately  or  jointly  as  we  wish.  In  doing  this,  it  should  bo  our  hope 
to  concentrate  the  effects  of  fitting  the  curve  to  which  the  data  is  compared 
as  thoroughly  --  and  into  as  few  comparisons  --  as  reasonably  may  be. 

In  larger  samples,  particularly  in  the  absence  of  grouping,  one  can 
go  a  long  way  toward  the  separation  of  "badness  of  fit"  into  three  parts; 

(al)  underestimated  badness  of  fit,  where  the  almost  inevitable 
fitting  of  parameters  has  concealed  any  true  badness, 

(a2)  systematic  badness ‘of  fit,  where  the  deviations  are  both 
interpretable  and  indicative  of  inadequacy  of  shape  of  model, 

(a3)  irregular  badness  of  fit,  often  an  indication  only  of  inadequacy 
of  simple  random  sampling  --  no  evidence  of  inadequacy  of  distributional 
shape. 

Once  this  is  accomplished,  the  introduction  of  the  ideas  underlying 
ordered  plotting  allows  us  to  break  new  ground,  to  --  reasonably  and 
sensibly  —  inquire  as  to  goodness  of  fit  for  many  kinds  of  nonrandom 
samples  without  preassumption  of  what  kind  of  nonrandomness  is 
involved.  Early  trials  of  such  techniques  have  had  quite  illuminating 
results  (Quandt  1964,  196  ?). 

These  are  only  the  beginning.  Deomnibusing  of  all  our  usual 
omnibus  procedures  will  do  much  to  occupy  both  technique -manufacturers 
and  philosophy  understander*  in  the  years  just  ahead. 

10,  THE  JACKKNIFE,  The  "jackknife "  procedure  allows  almost 
any  of  us  to  set  approximate  confidence  limits  on  almost  any  results 
calculated  from  data  which  go  a  reasonable  way  toward  revealing  the 
variation  whose  likely  effects  are  to  be  spanned  by  the  confidence  interval. 
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In  it*  simplest  form,  the  jackknife  procedure  assumes  that 

(al)  we  have  data,  and  a  fixed  procedure  for  extracting  an  interest¬ 
ing  number  (or  numbers)  from  the  data, 

(a2)  this  procedure  can  be  applied  to  varying  amounts  of  data, 

♦ 

(a3)  the  data  can  be  divided  into  r  "pieces"  of  roughly  equal  "siee", 

(a4)  this  can  be  done  in  such  a  way  as  to  make  the  differences 
from  piece  to  piece  "adequately  reflect"  the  sorts  of  variation  whose 
effects  are  to  be  spanned  by  the  confidence  interval, 

(a5)  the  prototype  case  of  "adequately  reflect"  is  the  sampling 
of  r  "pieces"  from  a  very  large  collection  of  pieces,  whose  combined 
processing  would,  by  definition  give  the  right  answer, 

(a6)  the  results  of  the  processing  are  not  narrowly  estimated,  in 
the  sense  that  no  one  piece  has  (and  no  very  few  pieces  have)  a  dominat¬ 
ing  effect  upon  the  result. 

Given  all  this,  to  some  reasonable  approximation  and  according  to 
some  reasonable  belief  (which  is  all  that  one  can  ever  truly  demand)  the 
analyst  treats  his  data  as  follows! 

(bl)  Let  y^  be  the  result  of  processing  all  the  pieces  of  data 
together. 

(b2)  Let  y^,  read  "y-not-i",  be  the  result  of  processing  all  but 
the  ith  piece  of  data  (hence  processing  r-1  out  of  r  of  the  pieces  together). 

(b3)  Let  y^,  read  "y-pseudo -i",  be  given  by 

y„i  •  ■•y.H  •  <■"%(!) 

(b4)  Let  y^  +  j'S^  be  the  mean  of  the  y^ ,  and  the  confidence 

interval  generated  by  a  naive  application  of  Student's  to  the  y^  (as  if 
they  were  a  sample). 
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The  procedure  is  simple,  the  Ann  rnvimfl H rm  4  m  itan«»1]Ur  ■  ■  •  f  *  C tc  Ty 

and  the  technique  is  applicable  in  very  diverse  and  complex  circumstance* 

Happily  this  technique  has  begun  to  receive  attention  from  some  of 
those  fitted  to  pinpoint  some  of  its  weaknesses  and  difficulties.  In 
particular,  there  has  been  inquiry  into  the  asymptotic  behavior  of  the 
technique,  especially  where  condition  (a6)  fails  (Miller  1964).  It  is  to 
be  hoped  that  there  will  be  more  such  studies  --  and  that  their  results 
will  be  correctly  evaluated  from  the  point  of  view  of  practice. 

The  cases  where  (a6)  is  most  likely  to  fail  are  those  in  which  a 
single  order  statistic,  a  median,  a  maximum,  a  minimum,  or  a  few 
order  statistics  play  an  unusually  important  part.  In  some  of  these, 
particularly  where  medians  and  other  inner  order  statistics  are  concerned 
we  have  other  means  of  assessing  the  stability  of  our  answers  that  are 
adequately  robust.  In  these  cases  we  should  clearly  use  these  alternate 
procedures. 

In  others,  often  those  involving  maxima,  minima,  and  ranges,  it 
is  clear  that  a  properly  assessed  uncertainty  for  the  quantity  of  interest 
will  inevitably  depend  on  such  matter*  as  the  actual  shape  of  the  under* 
lying  distribution  or  distributions.  Here  robustness  is  impossible,  and 
so  is  certainty  of  validity  for  any  confidence  procedure.  It  will  often  be 
true  that  the  best  that  we  can  do  is  to  use  the  jackknife  in  such  situations, 
even  though  we  know  it  may  be  fallible.  It  is  usually  better  to  have  some 
idea  of  the  uncertainty  of  our  values  rather  than  none,  (No  confidence 
interval  will  ever  be  computed  from  data  in  such  a  way  as  to  include  all 
possible  sources  of  variation,  eince  no  body  of  data  allows  all  possible 
source*  to  reveal  themselves.  A  little  more  inadequacy  will  not  be 
fatal.) 

In  cases  where  (a6)  is  not  in  question,  the  situation  is  rather  similar, 
If  there  is  available  a  robust  special  confidence  procedure  clearly 
applicable  to  the  case  at  hand,  by  all  means  use  it.  Otherwise  use  the 
jackknife. 

11.  ESTIMATED  VARIANCES  FOR  WEIGHTED  MEANS. 

(a)  Gwen  n  uncorrelated  observations  y  with  the  same  average 
value  and  fixed  finite  variances  ' 


for  which  also 
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2  /  - 

ii  an  unbiased  estimate  of  the  variance  ff  /n  of  y. 

It  is  known,  but  not  to  enough  people,  or  clearly  enough,  that  (b)  has. 
no  part  in  the  relation 

2  — 

(*)  ave  Sj  ■  var  y 


which  i*  a  consequence  of (a)  alone, 

We  have  here  a  simple  example  of  an  umbra -penumbra  situation  in 
which  two  models,  one  encompaising  the  other,  are  wisely  considered 
simultaneously,  The  penumbra  or  outer  model,  here  defined  by  (a) 
above,  suffices  for  the  validity  of  c~  as  an  estimated  variance  y  (in  the 
sense  that  (*)  then  holds).  The  umbra  or  inner  model,  here  defined  by 
(a)  and  (b)  together,  ensures  the  optimality  of  s*  as  the  unique  quadratic 
function  of  the  that  (i)  satisfies  (*)  and  (ii)  mfhimises  its  own  variance 
among  the  quadratics  that  do  this, 

The  pattern  here:  "validity  in  the  outer  model,  optimality  in  the 
inner"  is  but  one  of  many  possible  patterns  for  simultaneous  model 
pairs.  It  is  however,  one  of  the  most  important  ones,  one  that  needs 
much  more  explanation, 


Deiign  of  Experiment* 


707 


Suppose,  for  instance,  that  our  concern  is  not  with  y  but  with 


£  c .  y. 
2/1 


9 


where  the  c  j>  0  are  fixed.  Can  we  use  the  values  of  the  to  determine 

a  quadratic  function  Q(y, ,  v  , .  .  .  ,  y  )  so  that 

1  '  c  n 


(»») 


ave  Q  =  var  y_ 


provided  only  that  (a)  holds  and  the  c.  are  as  assumed?  CerttUnly  We 
can  do  this.  We  can,  indeed,  press  right  on,  and  find  a  Q  which  (i) 
satisfies  (**)  under  (a)  and  (ii)  minimises  its  own  variance  under  (a)  and 
(b)  combined.  Nay  more,  we  may  replace  (b)  by 


(c) 


the  variances  of  the 


are  in  known  ratio, 


in  that 


var  y. 

'i 


V 


2 


where  the  are  fixed  and  known. 

For  each  choice  of  [dj  ,  there  will  be  a  Q  satisfying  (*+)  under  (a) 
and  minimising  var  Q  under  (a)  and  (c).  This  Q  will,  in  fact,  be 
different  for  different  choices  of  ^d^j  . 

We  could  write  down,  in  closed  form,  expressions  for  thsss  Qls, 
but  their  detailed  form  is  of  far  less  concern  to  us  than  ths  facts  that 

(dl)  we  can  have  any  of  many  umbras  with  a  single  penumbra 

(d2)  which  umbra  we  choose  can,-  sometimes,  turn  efficiencies 
topsy-turvy  without  affecting  validity 

(d3)  the  equally  weighted  mean  seems  to  have  no  unusual  roles; 
it  appears  to  be  just  another  weighted  mean,  the  one,  perhaps,  for 
which  certain  formulas  look  simplest. 
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We  need,  and  axe  inevitably  going  to  get  for  ourselves,  a  very  much 
wider  collection  of  instances  where  the  jtcr.c;?  umbra-penumbra 

model  pairs  have  been  worked  out,  much  to  our  illumination  and  advantage. 


II 

THE  RELATION  OF  EXAMPLES  TO  PRINCIPLES 

12.  HOW  THE  EXAMPLES  ILLUMINATE  THE  TWO  MAJOR  ASPECTS, 
The  first  (erector- set)  principle  is,  according  to  Table  1,  illustrated  by: 

(al)  newer  approaches  to  typical  values',  where  Winsoriring  is 
combined  with  Student's  t;  where  techniques  developed  for  single  samples 
are  expected  to  be  used  directly  or  indirectly  in  simple  and  multiple 
regression  and  in  all  sorts  of  analyses  of  variance  involving  replication 
within  cells, 

(a2)  new  dissections  of  factorial  tables;  where  we  try  to  use  both 
factorial  and  idosyncratic  dissections  at  the  same  time;  where  we  expect 
to  build  each  new  kind  of  diseection  into  more  and  more  complex  patterne, 

(a3)  unrestrained  monotone  transformation:  which  is  rapidly 
propagating  itself  in  cooperative  combination  with  a  wide  variety  of 
other  techniques, 

(a4)  internally  estimated  variances  for  weighted  means:  where  we 
learn  how  to  do,  knowingly,  for  weighted  means  what  we  have  so  long 
done,  often  unknowingly,  for  equally  weighted  means. 

While  notone  of  these  is  as  striking  as  Cuthbert  Daniel's  unpublished 
injections  of  2$'m  fractional  factorial  analysis  into  the  calculations  of 
multiple  regression,  or  ai  striking  as  the  technique-combinations  that 
are,  in  practice,  appropriate  to  a  variety  of  complex  bodies  of  data, 
they  do  offer  solid  illustrations, 

The  second  principle  (computation  cheaper,  more  used,  and  more 
vital)  is  certainly  well-exemplified  above,  Consider: 

(bl)  spectrum  techniques:  where  hand-calculator  work  would  be 
worthwhile  for  some  of  the  most  crucial  instances,  but  where  the  cost 
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of  hand  computation,  if  it  had  to  be  paid,  would  keep  us  from  many  of 
the  useful  and  illuminating  studies  we  actually 

(b2)  unrestrained  monotone  transformation;  where  iterative 
computer  algorithms  lead  us  easily  to  ends  not  at  all  reasonably 
accessible  by  hand  computation. 

(b3)  new  dissections  of  factorial  tables;  where,  when  the  still 
more  effective  techniques  arrive,  their  feasibility  will  depend  greatly* 
perhaps  absolutely,  upon  the  availability  of  computer!. 


(b4)  ordered  plotting:  where  much  of  the  real  push  forward  seem* 
to  be  associated  with  the  use  of  modern  computer  both  to  calculate  and  1 

to  make  such  plots.  j 

( b 5 )  deomnibusing:  where,  while  the  dsomnibusing  of  goodness  ! 

of  fit  is,  in  many  instances,  feasible  without  a  modern  computer,  it  f 

is  the  availability  of  computer  procedures  that  will  make  such 

techniques  popular.  , 


(b6)  the  jackknife:  where  one  of  the  great  advantages  la  that  a  . 
well-written  computer  problem  to  do  y^  can  also  be  used  to  do  y^^  , 

so  that  the  cost  of  jackknifing  is  only  a  little  more  running  time,  but 
no  extensive  effort  in  programming  or  debugging. 

While  these  are  matters  of  technique  manufacture  rathsr  thaii 
technique  use,  many  of  the  new  approaches  to  typical  values  are  only 
accessible  because  of  the  modern  computer  (as  when  Monte  Carlo 
techniques  are  required  to  find  critical  values,  even  when  the  under- 
*  lying  distribution  is  Gaussian). 

There  can  be  little  doubt  of  the  importance  of  the  second  principle, 

‘  13'  HOW  THE  EXAMPLES  ILLUMINATE  THE  FIVE  AREAS  07 

RAPID  EXPANSION.  The  first  area  where  rapid  expansion  is  trying 
to  repair  past  deficiencies,  principle  3,  involves  graphicality  and 
informality,  where  the  graph  is  used  &■  an  effective,  but  very  informal, 
way  of  connecting  the  data  to  the  human  judgment*  that  are  going  to 
be  made  about  it,  that  constitute  the  reasons  for  its  analysis,  Here. 
Table  1  points  out; 
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(al)  spectrum-like  analysis:  where  a  far  larger  share  of  judgments 
than  many  might  suppose  are  in  fact  based  on  crAnMcii  prcscr.Cati«u« , 
informally  examined. 

( a 2 )  ordered  plotting;  where  a  formal  significance  testing  procedure 
is  largely  replaced  by  an  informal  judgment  made  by  those  who  look  at 
the  plot. 

(a3)  hanging  rootograms:  where  we  have  striven  to  learn  graphically 
and  far  leea  formally  by  eeeking  an  approach  to  goodneaa  of  fit  where  a 
moderately  wiee  man's  eye  will  tell  most  of  the  story, 

The  second  area,  principle  4,  involves  graphically  and  incieiveneae 
and  la  quite  diatinct  from  the  first,  although  the  areas  share  graphically 
and  appear  together  in  many  techniques,  At  issue  here  is  that  grand 
property  of  many  graphs:  revelation  of  the  unexpected  through  the 
simultaneous  revealabillty  of  many  possible  deviations  from  neutrality. 
Table  1  directs  our  attention  to: 

(bl)  spectrum-like  technique*:  where  little  peaks  have  often 
revealed  new  phenomena,  as  in  Munk  and  Snodgrass  1957,  or  as  in  the 
detection  and  evaluation  of  the  natural  modes  of  vibration  of  the  earth: 

(b2)  unrestrained  monotone  transformations:  where  the  graph  of 
the  final  monotone  transformation  is  often  quite  revealing:  where  the 
structures  resulting  from  multidimensional  scaling  often  show  unexpected 
properties. 

(b3)  ordered  plotting:  where  w*  have  learned  that  half-normal  plots 
expose  many  kinds  of  interesting  behavior  other  than  the  stray  large 
values  to  detect  which  the  plot  was  invented  (Daniel  1959). 

The  third  area,  principle  5,  involves  flexibility  and  fluidity  and 
deserves  discussion  in  two  rather  separate  plecte.  Flexibility  here 
refers  to  the  existence  of  a  wider  variety  of  framework*  for  analysis 
and  inference,  thus  offering,  on  the  average,  a  better  match  to  the 
need*  of  the  problem.  Fluidity  refers  to  the  ability  of  single  analytical 
procedure*  to  respond  in  a  very  wide  variety  of  way*  to  the  apparent 
character  of  individual  bodies  of  data,  Clearly  a  continuous  graduation 
from  flexibility  to  fluidity  is  possible  -•  indeed  many  stages  along  this 


fsifljaaO—i  titt.  te  **  fcs*  Uss-  ^  it  *«*:  to*ry.  to-.  • 

<S©fc*  Sfeft  41  toe IHWWI &&&S: .toy ^fvfrnd e  SfftKe  analyst’  and 

; ;  m.a4»«c  <fc*  4Mb 

W«.  &»«*  ftfit  4ft**  vpmch  liability  dfrfe£t*-y  in  our- 

e*&MS®4#.»r  sgss«st  of  »iafSNrK3^  $&  $ffctirectly ,  -tHus:  '' 

(cl)  Our  xW*W«r  ap^ffc&sJWsS  typi-cal  values  are  not  yet  focused 
iHt©<PT»  form  --  ae  tfaey  roig&fc  aottf#  <$&y  be',  where  they  completely 
fiui4ie*4.  ([As  I  trust  tfe^y  never  will  be,  s-ipce  I  expect  *hat  the  analyst 
will  always  'be  very- often  able  to  add  irrso hmat-ron  about  the  underlying 
distribution  over  and  above  that  cafstataec!  in.  a.  single  small  sample.  ) 

Shall  we  use  trimmed  means,  Wir.eorifc'ed  means,  or  Hodges -Lehmann 
median  differences  ?  If  we  trim  or  Winsorize  ,  how  far  ?  We  have  not 
yet  provided  the  user  with  the  information  most  helpful  in  choosing 
answers  to  these  questions,  but  we  have.  begun  to  provide  him  the  flexible 
kit  of  tools  from  which  to  choose. 

(cE)  There  will  be  more  than  one  choice  among  new  dissections  of 
factorial  tables. 

(c3)  One  can  rightfully  say  that  the  modern  phase  in  spectrum-like 
analysis  comes  from  expanding  our  kit  of  tools  beyond  the  serial 
correlation  function  and  the  periodogram  (neither  of  which  was  really 
helpful). 

(c4)  The  jackknife  is  a  great  aid  to  flexibility;  in  most  situations 
it  removes  that  grim  complaint  'fBut  if  we  do  that  how  can  we  compare 
the  result  with  chance  fluctuations  ? n  and  allows  much  freer  choice  of 
technique. 

(c5)  The  ability  to  estimate  variability  for  all  weighted  means 
with  the  same  robustness  as  for  the  equally  weighted  ca.se  is  a  similar 
contributor  to  flexibility. 

For  the  moment,  the  unrestrained  monotone  transformation  is  the 
outstanding  example  of  complete  fluidity.  We  face  a  challenge  in 

finding  ot  he  rs  . 

The  fourth  area  of  growth,  principle  (6),  involves  the  empirical 
t'u. very  of  techniques  .  as  opposed  to  their  theoretic© -mathematical 
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discovery.  Here  again  wo  deal  with  a  matter  of  amount  rather  than  kind. 

.It  seems  likely  that  no  technique  was  developed  solely  empirically,  without 
?<any  "theoretical"  insight  at  all,  though  many  have  been  developed  without 
any  trace  of  a  mathematical  mode.  At  the  other  extreme,  techniques 
baaed  on  rigid  mathematical  models,  clearly- specified  criteria,  and 
vigorous  optimization  only  gain  credibility  from  some  emp’”tcal  support, 
whether  of  their  hypotheses  or  of  their  functioning  in  practice. 

Accordingly,  our  examples  will  tend  to  be  ones  that  show  a  greater 
empirical  content  than  most,  ones  whose  developments  are  separated  in 
amount,  rather  than  kind,  from  those  of  most  techniques.  Table  1  cites: 

(dl)  newer  approaches  to  typical  values;  where  "trimming"  and 
"Winsorization"  came  into  being  at  least  as  much  because  of  how  they 
worked  in  practice  as  for  any  insight  or  theoretical  argument:  where 
the  matching  of  denominators  to  numerators  has  come  about  by  empirical 
comparisions  based  on  tables  of  order  statistic  moments;  where  the 
critical  values  have  often  to  be  determined  by  Monte  Carlo. 

(d2)  spectrum-like  techniques:  where  one  source  of  modern  lag- 
windows  was  Hamming's  observation  that  the  points  of  an  estimated 
spectrum  for  a  single  particular  set  of  data  would  be  improved  by 
hanning;  where  the  pseudoautocorrelation  was  suggested  by  a  diffuse 
analogy  with  the  cepstrum,  and  only  the  fact  that  it  seemed  to  work  made 
it  plausible. 

Though  not  quite  a  technique  of  data  analysis,  the  near  coristancy 
of  standardized  5%  distances  for  Pearson  curves  (Pearson  and  Tukey 
196?)  is  based  upon  Charles  P.  Winsor’s  wholly  empirical  discovery 
of  the  near  constancy  of  the  standardized  5%  distance  for  chi-square. 

The  fifth  growth  area,  principle  7,  is  one  of  focusing  and  parsimony. 
Some  books  on  probability  and  statistics  reveal  that  every  sample  (or 
other  grouping  of  observations)  is  unusual  in  some  way.  (if  only  by  how 
closely  it.  matches  a  copy  of  itself.  )  It  is  rare,  however,  that  the 
discussion  carries  on  to  the  logical  conclusions:  First,  that  it  is 
important  to  be  restrictive  in  the  kinds  of  unusualness  to  which  one 
pays  attention.  Second,  that  one  escapes  this  difficulty  when  one  can 
focus  all  one's  attention  upon  a  single  numerical  aspect,  or  on  a  very 
few  numerical  aspects.  Third,  that  once  a  fair  number  of  such  aspects 
are  involved  one  is  in  a  situation  very  like  the  unrestricted  case  and 


Design  of  Experiments 


that  just  how  one  divides  his  attention  is  of  great  importance.  One  can 'be 
wisely  parsimonious  with  one  of  one*s  most  valuable  possessions,  by 
focusing  one's  attention  where  this  is  most  likely  to  be  profitable. 

Table  1  directs  our  attention  to: 

(el)  new  dissections  of  factorial  tables:  where,  instead  of  merely 
giving  a  single  number  to  an  inchoate  mass  of  "interaction",  we  are 
striving  to  attend  to  very  particular  aspects,  such  as  the  single  very 
unusual  cell  or  indications  that  some  other  mode  of  expression  will  lead 
to  a  better  approximation  to  additivity. 

(e2)  deomnibusing:  in  each  of  whose  specific  instances  we  are  try¬ 
ing  to  improve  our  focusing,  to  learn  about  something  identifiable  and 
thereby  to  increase  both  the  value  of  our  knowledge  and  the  chance  of 
gaining  it. 

14 .  HOW  THE  EXAMPLES  ILLUMINATE  THE  AREAS  OF  INCREASED 
ATTENTION.  The  first  principle  of  increased  attention,  principle  8; 
calls  for  greater  attention  to  being  approximately  right  rather  than 
exactly  wrong.  The  hardest  part  of  this,  at  least  for  the  mathematician, 
is  to  admit  that  one  is  proceeding  approximately  --  even  though  it  is  hard 
to  see  how  one  can  ever  do  better  in  the  real  world. 

Table  1  directs  our  attention  to: 

(al)  new  dissections  of  factorial  tables:  where  we  are  seeking  to 
ask  the  questions  of  greatest  importance  to  us,  even  though  their  asking 
tends  to  destroy  the  neat,  nice,  manageable,  null  hypothesis  which  was 
the  formal  foundation  for  the  classical  asking  of  less  useful  questions; 
where  our  conclusion  levels  are  going  to  become  approximate;  where 
there  will  be,  for  a  period  of  years  at  least,  no  formal  driterion  to 
insulate  us  from  the  very  real  difficulties  of  picking  a  good  technique. 

(a2)  deomnibusing:  where  we  are  again  very  willing  to  be  approxi¬ 
mate  in  the  answering  of  more  meaningful  questions. 

(a3)  the  jackknife:  where  by  admitting  that  an  approximate  con¬ 
clusion  procedure  can  serve  us,  we  have  brought  a  very  much  wider 
range  cf  techniques  into  the  fold  for  which  confidence,  and  significance, 
statements  are  at  hand  for  use  when  appropriate. 
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The  8»ccnd!. principle  of  increased,  atten’tiorv,  principle.  9,  calls  for 
more  use  of  model-pairs  and  other-  ’’it  might  be  A  and  it  might  be  B 
and  we  must. think  about  both  together"  approaches.  The  use  of  pairs 
of  models  as  alternatives,  as  in  the  Neyman-Pearson  account  of  hypothesis 
testing,  is  classical.  (Pearson  (1939)  points  out  how  much  Student  had 
to  do  with  the  recognition  of  its  importance.  )  It  is  remarkable,  by 
contrast,  how  little  attention  has  been  paid  to  pairs  of  models  simulta¬ 
neously  considered.  Perhaps  this  is  because,  in  many  instances ,  the 
use  of  simultaneous  model  pairs  inevitably  attracts  attention  to  the 
deficiencies  of  a  technique.  .  In  an  optimality- validity  umbra-penumbra 
situation,  for  example,  emphasizing  the  validity  of  the  technique  in  the 
penumbra  cannot  help  reminding  us  that  it  is  not  optimum  throughout. 

Table  1  dravys  our  attention  to; 

(bl)  newer  approaches  to  typical  values,  where  Gaussian  and  crudely 
Gaussian  underlying  distributions  provide  umbra  and  penumbra  that  are 
used  in  varied  ways:  .  relatively  good  efficiency  for  the  Gaussian  and 
validity  for  all  symmetric  distributions;  critical  values  set  for  the 
Gaussian  (and  approximately  valid  elsewhere)  and  moderately  high 
efficiency  (except  for  unseizable  opportunities)  anywhere  near  the 
Gaussian:  etc.  •  .  , 

(b2)  internally  estimated  variance  for  weighted  means;  where  the 
whole  discussion  is  on  an  umbra -penumbra  basis. 

The  third  principle  of  increased  attention,  principle  10,  calls  for 
making  the  relation  of  estimator  and  estfmand  a  two-way  street. 

(See  Tukey  1962,  p.  10  and  references  cited  there.  )  The  mathematician 
wants  the  problem  to  come  before  the  solution.  But  a  good  solution  can 
often  be  recognized  as  such  before  we  have  identified  one  or  more  of  the 
problems  it  solves.  And  a  good  solution  may  be  good  because  it  solves 
a  problem  other  than  the  one  as  whose  solution  it  is  customarily  derived. 

Table  1  directs  our  attention  to: 

(cl)  spectrum-like  techniques:  where  much  has  been  gained  by 
asking  what  spectrum  estimates  actually  do  estimate,  rather  than  by 
asking  for  asymptotic  results  which  demand  unreal  amounts  of  data, 
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(c2)  the  jackknife:  wnere  the  esiim«i.ur  la  defined  by  -  prcc*!«, 
selected  by  what  wisdom  the  analyst  possesses,  and  the  estimand  follows 
after  it,  like  the  tail  of  a  kite. 

In  each  of  these  three  areas  of  increased  attention,  if  one  goes 
through  the  uncited  examples  carefully,  one  will  find  each  principle 
recurring  again  and  again,  though  usually  less  explicitly.  If  one  looks 
at  the  three  areas  in  the  right  way,  they  seem  to  blur  and  move  together 

into  one. 

If  we  look  at  all  the  principles,  the  same  blurring  appears,  though 
not  as  obviously,  There  ia  a  sense  in  which  all  these  principles  are 
"sisters  under  the  skin". 


in 

THE  CONCLUSION 

15.  SUMMARY.  If  we  ask  of  the  near  future  of  processes  of  data 
analysis,  one  can  predict  three  essentials: 

(dl)  greater  realism, 

(d2)  greater  effectiveness, 

(d3)  greater  use  of  computers. 
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MONTE  CARLO  TECHNIQUES  TO  EVALUATE 
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M.  M,  Everett,  D,  L.  Colbert,  and  L,  W.  Green,  Jr, 
Pratt-Whitney  Aircraft 
Florida  Research  and  Development  Center 
West  Palm  Beach,  Florida 


1.  INTRODUC TION.  Monte  Carlo  eimulation,  using  today's  high 
speed  computers,  has  opened  new  fields  in  analysis  of  systems  previ¬ 
ously  unavailable  to  engineers  and  mathematicians.  Especially  valuable 
are  the  approaches  they  offer  to  study  the  accuracy  and  precision  of 
some  of  the  empirical  relationships  used  in  analysis  of  variance  and 
experimental  design. 

This  report  gives  the  results  of  two  such  simulation  programs 
made  at  Pratt  &  Whitney  Aircraft's  Florida  Research  U  Development 
Center,  It  is  not  the  purpose  of  this  paper  to  present  the  findings  of 
these  studies  as  absolute  truisms.  They  are,  however,  provided  as 
the  results  of  case  histories  and  do  offer  a  method  for  further  explora¬ 
tion  of  analytical  solutions  in  the  field  of  analysis  of  variance  and 
experimental  design. 

The  two  simulations  presented  here  are  Rejection  Criteria  for 
Approximate  Student's  "t"  Test,  and  Bias  in  the  Analysis  of  Variance 
Components  from  an  Unbalanced  Design, 

2.  DISCUSSION. 

A,  REJECTION  CRITERIA  FOR  APPROXIMATE  STUDENT'S  "T"  TEST 


One  of  the  simplest  designed  experiments  is  that  designed  to  test, 
or  compare,  the  first  momenta  of  two  lots  or  populations^,  Of  interest 
here  is  the  case  of  the  comparison  of  two  means  (5?  and  Y)  calculated 
from  the  samples  drawn  from  those  populations  when  the  population 


variances,  r  <s  , 
x  y 


are  net  equal. 


This  paper  reports  on  an  investigation  of  three  different  commonly 
used  methods  to  determine  critical  values  for  this  situation,  The  pur¬ 
pose  of  the  investigation  was  to  compare  the  relative  merits  of  each, 
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assuring  that  the  true  level  of  significance  was  at.least  as  great  as  the 
pre-selected  level  of  significance,  and  to  obtain  an  unbiased  estimator 
having  a  minimum  variance. 

To  compare,  by  a  "t"  test,  two  means  from  independent  samples, 
where  it  is  suspected  or  known  that  the  variances  are  unequal,  the  test 
would  be: 


(a) 


cal 


(X  -  Y)  -  (|1X  -  |!y) 
s  2/N  +  s  Z/N 

x  x  y  y 


When  the  hypothesis  to  be  tested  is  that  =  |±^,  this  reduces  to 


(b) 


cal 


V  2 


s  /N  +  s  /N 
x  x  y  y 


2  2 

In  the  case  where  c r  /  tr  ,  t  does  not  follow  the  student's  "t" 

x  y  cal 

distribution  with  N  +  N  -2  degrees  of  freedom.  Therefore  some 

x  y 

critical  criterion,  such  as  a  modified  t-distribution,  must  be  used  to 
judge  significance. 

1.  Methods  Used  to  Determine  Critical  Values. 

Method  1  -  Cochran  and  Cox  Approximation 


2  2  . 

s  s 

—  t  +-£  t 
N  ax  N  ay 

X _ _ _ jr _ ' 


*'.«  * 


N 


N 


x 


*  * 
i 


This  method  utilizes  a  weighted  mean  of  the  tabular  t  values  for  the  two 
samples. 
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Ivic  l  juju 


n:  .. . 


J/cs 


..  A 
7  -  *rr 


a  r>r» 


t1  (2)  =  tabulated  value  of  student's  "t"  associated  withy  degrees 
of  freedom  where 


This  approximation  assumes  that  the  mean  comparisons  follow  a 

student's  "t"  distribution  not  at  (N  +  N  -2)  degrees  of  freedom  but 

x  y  • 

rather  at  some  y  degrees  of  freedom, 

Method  3  -  Satterthwaite -Welch  Approximation 

t1  (3)  «  tabulated  value  of  student's  "t"  distribution  with  y  degrees 
of  freedom  where 


This  is  the  approximation  for  the  modified  degrees  of  freedom.  It  is 
shown  in  various  texts  in  different  algebraic  forms. 

In  determining  (2)  and  t^  (3)  it  is  not  necessary  to  round  the 

degrees  of  freedom  to  the  lower  value;  the  tables  can  be  Interpolated 
for  an  unbiased  estimate.  However,  in  tables  I  and  II,  discussed  later, 
for  t^  (2)  and  (3)  the  lower  rounded  degrees  of  freedom  were  used  or 

the  percentiles  shown  would  have  been  somewhat  smaller, 
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2.  Simulation  Procedure 

These  three  methods  were  compared  by  Monte  Carlo  simulation  on  the 

IBM  7090  and  1620  computers.  For  a  stated  set  of  parameters  and  sample 

size,  10,  000  samples  were  drawn  from  both  the  X  and  the  Y  populations. 

These  samples  were  randomly  paired  and  their  first  moments  input  to 

equation  (b).  The  output  was  10,000  values  of  t'  .,  under  the  restriction 
__  cal 

that  E(X  -  Y)  =  0.  Then  all  10,000  values  were  ranked  and  the  percentiles 

identified.  This  process  was  repeated  for  various  combinations  of 

,  <r^,  N  ,  and  N  . 
x  y  x  y 

Approximate  rejection  values  for  t'^  (l),  t^  (2),  and  t^  (3)  were 

calculated  and  averaged  for  a  at  levels  of  90%,  95%,  and  99%.  The 
rejection  values  were  then  compared  to  the  appropriate  percentile  level 
from  the  ranked  values. 

3,  Conclusions  and  Discussion. 


Tables  I  and  II  summarize  the  test  cases  that  were  simulated.  The 
recorded  levels  for  the  t^  ^  are  the  average  estimates  for  the  actual 

levels  90%,  95%,  and  99%.  Table  I  is  used  to  demonstrate  the  output  of 
the  simulation  process.  It  compares  the  three  methods  when  used  with 
equal  variances.  The  only  significant  conclusion  demonstrated  is  that 
the  Cochran  it  Cox  approximation  is  an  estimator  whose  confidence  level 
is  at  least  as  high  as  the  prior  selected  confidence  level.  Table  II 
continues  to  demonstrate  this. 

If  a  bias  exists  in  Method  2  and  Method  3  It  does  so  only  at  certain 
levels  of  the  parameters  and  their  sample  sizes,  This  indicates  that  an 
interaction  of  the  variables  exists.  Table  II  compares  only  a  few 
situations  and  is  entirely  too  general  to  draw  many  exact  conclusions, 

It  does,  however,  give  an  insight  into  the  comparative  accuracies  involved. 

In  additional  simulation  studies  it  has  been  found  in  all  cases  observed 
that  the  5atterthwaite-Welch  Method  was  a  more  precise  estimator  than 
the  Dixon  and  Massey  approximations.  Further  studies  are  required  to 
obtain  an  unbiased  estimate  with  a  minimum  variance. 
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It  may  be  noted  that  an  exact  solution  due  to  H.  Scheffe  has  been 
omitted  from  this  study.  Scheffe's  method  is  based  on  the  fact  that  if 
n  <  n_,  a  sample  of  size  n.  may  be  randomly  selected  from  the  larger 

X  £  X 

sample  size  of  size  n  .  It  is  then  possible  to  calculate  Scheffe's  "t" 

*-«'■  *  w 

statistic  by: 


which  is  distributed  as  Student's  "t"  with  n  -1  degrees  of  freedom.  It 
is  immediately  obvious  that  the  relative  information  of  this  statistic 
decreases  as  the  value  of  n2_n^  becomes  large,  since  n^-n^  observations 

are  randomly  eliminated  from  the  calculation  of  the  "t"  statistic  under 
the  assumption  that  n^  <  n^. 

Since  this  loss  of  information  is  especially  severe  for  the  case 
where  one  sample  size  is  very  small,  this  method  was  not  considered 
in  this  study. 

B.  BIAS  IN  THE  ANALYSIS  OF  VARIANCE  COMPONENTS  FROM  AN 
UNBALANCED  DESIGN. 

It  was  suspected  that  a  bias  existed  in  the  estimates  of  the  compo:- 
nents  of  variance  when  analysis  of  variance  techniques  are  applied.  It 
was  further  suspected  that  this  bias  was  due  to  unequal  sample  sizes. 

If  this  bias  could  be  related  to  sample  sizes  and  sample  size  ratios  then 
it  may  be  possible  to  derive  an  unbiasing  technique,  Using  the  IBM 
7090  computer,  a  Monte  Carlo  Simulator  was  written  to  determine  if 
this  suspected  bias  existed  and  to  study  the  possibilities  of  finding  a 
method  to  identify  this  bias. 
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1.  Simulation  Procedure 

The  simulator  was  designed  to  determine  the  distribution  of  estimates 
of  process  variance  from  a  one-way  ANOVA,  byproduct  of  which  were 
estimates  of  the  within-process  variation  discussed  below.  To  simulate 
the  two  sources  of  variation,  two  populations  of  normally  distributed 
random  numbers  were  set  up.  The  means  of  these  distributions  were 
given  fixed  arbitrary  values,  while  the  standard  deviations  (and  therefore, 
the  variances)  were  variable.  The  first  population  was  designated  as 
the  process-to-process  source  of  variation.  The  second  was  designated 
as  the  within-process  source  of  variation.  It  was  decided  that  three 
ratios  of  standard  deviations  of  these  populations  would  be  used.  There 
were: 


rpr°ce?°-t°-i;rocess  pf  Q  0  2.  Q  . 

<s  within-process 

These  values  were  selected  because  they  cover  the  general  area  of 
interest  in  estimating  process-to-process  variation.  It  was  further 
decided  that  for  each  ratio  above,  a  control  case  (balanced  data)  should 
be  run,  in  addition  to  a  case  with  mild  unbalance,  and  a  case  with 
extreme  unbalance.  The  balanced  case  had  four  runs  with  10  data 
points  for  each  run.  The  mildly  unbalanced  case  had  four  runs  with 
8,  9,  10,  and  12  data  points  each.  The  extremely  unbalanced  case  had 
four  runs  with  5,  3,  10,  and  15  data  points  in  each.  The  analysis  of 
variance  described  above  was  carried  out  for  each  case  on  an  IBM  7090 
computer  in  the  following  sequence: 

1.  Four  values  were  selected  from  the  population  of  process-to- 
process  random  numbers.  One  of  these  corresponds  to  each 
process. 

2,  Four  sets  of  numbers  were  selected  from  the  within-process 
population.  Each  of  these  sets  corresponded  to  one  of  the  four 
processes.  For  example,  for  the  control  case,  each  set  would 
have  ten  random  numbers;  for  the  extremely  unbalanced  case, 
the  set  corresponding  to  the  first  process  would  have  five 
members,  the  second  set  would  have  three,  etc.  The  value, 
selected  in  step  1,  for  the  first  process  is  added  to  each  member 
of  the  set  of  within-process  numbers  for  the  first  process;  the 
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second  process  number  is  added  to  each  value  of  the  aecond  set, 
etc.  In  this  manner  an  array  of  numbers  is  produced.  Each 
column  has  a  common  process  effect  while  within  each  column 
there  is  a  within-proce ss  effect. 

3,  The  formulas  of  the  analysis  of  variance  were  ueed  to  estimate 
the  process-to-process  and  within-process  variances.  These 
values  are  stored  in  the  computer. 

4,  Steps  1-3  are  repeated  1000  times  ao  that  the  1000  estimates  of 
each  variance  are  obtained. 

5.  The  1000  values  are  then  ranked  and  the  mean,  standard  dev¬ 
iation,  and  standard  error  of  the  mean  are  computed.  The 
ranked  estimates  and  computed  statistics  are  then  listed. 

6.  The  plot,  figure  1,  was  then  made,  showing  the  frequency 
distribution  of  the  estimates. 

For  comparison  purposes  the  plots  of  the  control  cast,  the  mild 
unbalanced  case,  and  the  extremely  unbalanced  case  art  shown  togsther 
in  figure  1  for  the  ratio  1.  0  to  1,  0  of  standard  deviations. 

The  result*  of  the  aimuiation  are  summariced  in  table  III,  If  a 
bias  exists  in  either  the  process-to-process  or  the  within-process 
variance  it  is  not  evident  here.  There  is  however,  a  relatively  large 
scatter  of  the  variance  estimates, 

2.  Conclusions  and  Discussion 

Based  on  the  Monte  Carlo  Simulator,  the  following  conclusions 
were  reached! 

1,  The  distributions  of  the  estimates  of  within-proce  ss  variances 
(discussed  in  the  appendix)  were  approximately  normal.  These 
distributions  exhibited  a  marked  central  tendency  and  a  degree 
of  eymmetry. 

2.  The  bias  in  the  estimates  of  within-process  variability  was 
negligible  in  each  case  tested.  The  cases  using  unbalanced 
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data  (unequal  sample  sizes  mn)  did  not  demonstrate 

biases  significantly  larger  than  the  control  case. 

These  first  conclusions  were  not  unexpected  and  were  more  or  less 
byproducts  of  the  simulation.  The  more  important  conclusions  follow; 

3.  The  distributions  (figure  1)  of  the  estimates  of  process-to- 
process  variances  were  highly  skewed  to  the  right  and  truncated 
on  the  left.  However,  the  cases  using  the  unbalanced  data 
were  no  more  skewed  than  the  control  case  (balanced  data), 

4.  The  bias  in  the  estimates  of  process-to-process  variability 
was  negligible  in  each  case  tested,  including  the  cases  of 
unbalanced  data. 

5.  Although  estimates  of  process-to-process  variance  resulting 
from  the  analysis  of  variance  technique  are  not  optimum,  no 
known  method  of  improving  this  situation  exists. 

The  last  three  conclusions  presented  represent  the  main  intent  of 
this  study,  It  must  be  noted  that  the  estimates  of  process -to-procSM 
variance  cannot  be  considered  "optimum"  estimates.  Since  an  optimum 
estimate  should  have  minimum  variance  and  minimum  total  error,  the 
estimates  based  on  analysis  of  variance  techniques  cannot  be  optimised, 
Any  attempt  to  further  optimize  them  through  the  use  of  an  unbiasing 
technique  must  reduce  the  variance  of  the  estimates  and,  at  the  same- 
time,  increase  the  relative  value  of  each  estimate  (because  of  skewness), 

In  figure  2  the  sample  curve  for  the  distribution  of  estimates  of 
process -to-process  variance  is  skewed  and  exhibits  a  large  amount  of 
scatter.  The  distribution  of  an  optimum  estimating  procedure  should 
have  minimum  variance  and  minimum  bias,  approached  by  the  second 
curve  shown  in  figure  2.  To  optimize  the  present  estimates  (obtained 
from  ANOVA)  of  process-to-process  variance,  the  scatter  of  the 
distribution  of  these  estimates  should  be  reduced.  To  accomplish  this , 
each  estimate  (s*)  should  be  divided  by  &  factor  K,  where  K  is  greater 
than  one  (1).  The  proper  selection  of  K  will  minimize  the  variance. 
However,  this  will  bias  the  estimate  so  that  a  correction  must  be  made; 
that  is,  the  mean  estimate  of  the  variance  will  be  only  l/K  of  the  true 
value.  Thus,  1  -  l/K  must  be  added  to  each  estimate  s^  to  unbias  the 
estimates,  However  for  any  estimate  s^; 
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(1/K)s2  +  (1-1/K)«2  =  «2/K  +  «2  -  s2/K  =  ;2  . 

Therefore,  for  any  K  selected  the  estimate  is  not  improved.  Any 
other  plan  to  optimize  these  estimates  will  fail  since  to  reduce  the  scatter 
a  bias  must  be  introduced  and  to  minimise  this  bias  the  scatter  must  be 
increased. 
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Ammunition  Proc  L  Supply  Agency 
Johns  Hopkins  Univ 
Ft,  Detrick 

Stat  Engrg  Lab,  Natl  Bur  of  Stds 
Natl.  Security  Agency,  Ft.  Meade 
Harry  Diamond  Labs 
US  Army  Strategy  b  Tactics  Analysis  Gp 

US  Army  Res  Office,  Durham  . 

US  Army  Natick  Labs 

Biometrical  Svcs,  ARS 

Science  It  Tech  Div,  Library  of  Congress 

WRAIR 

Ft.  Meade 

Stanford  Univ 

US  Post  Office  Dept 

Bur  of  Supplies  L  Accts 

Oklahoma  Stat  Univ 

Bu  of  Ships,  Navy  Dept 

Frankford  Arsenal 

US  Army  Strategy  b  Taetlcs  Analysis  Gp. 

NC  State  College 

Office  of  Naval  Re s 

Springfield  Armory 

Atlantic  Res.  Corp. 

US  Army  Strategy  U  Tactics  Analysis  Gp, 
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Malligo,  Mr.  John  E, 
Maloney,  Dr.  Clifford  J. 
Mandelson,  Mr,  Joseph 
Mann,  Dr.  Henry  B. 
Manthei,  Mr.  James  H. 
Martin,  Mr.  Francis  F. 
Masaitis,  Dr.  C, 

Matthews,  Mr.  Gerald  A. 
Matthis,  Mr.  Carlton  L. 
Mauss,  Mrs.  Basse  Day 
McCormick,  Mr.  Garth 
McGroddy,  Miss  Patricia 
McIntosh,  Mr.  Albert  L, 
McIntosh,  Dr.  Wm,  B. 
Menken,  Mrs.  Jane  A. 
Miller,  Mrs.  Christine 
Mood,  Dr.  A.  M. 

Moshman,  Mr.  Jack 
Moss,  Mr.  David  M. 

Moss,  Mr.  H.  Donald 
Myera,  Mr.  Raymond  K. 
Mylander,  Mr,  W,  Charles 
Nassimbene,  Mr.  Raymond 
Natrella,  Mrs.  Mary  G. 
O'Connor,  Mr.  Desmond 
Oklln,  Prof  Ingram 
Olon,  Mr.  Frederick  A. 
Orleans,  Beatrice  S. 

Pabst,  Dr.  Wm.  R.  ,  Jr. 
Panos,  Mr.  Robert 
Parrish,  Dr,  Gene  B. 
Parvin,  Mr.  David  W.  ,  Jr. 
Parson,  Dr.  Emanuel 
Pefcper,  Mr.  Leonard 
Perry,  Virginia  W, 
Persweig,  Mr,  Michael 
Pettigrew,  Mr.  Hugh 
Pfeiffer,  Mr.  Otto  H, 
Piepoli,  Mr.  Carl  R, 
Podolsky,  Mr.  Benjamin 


Ft.  Detrick 

NTH 

Dir.  of  Quality  Assurance,  Edgewood 

Math  Res  Center,  USA,  UnivofWis 

CRDL,  MED  RES,  Edgewood 

Booz  Allen 

BRL-Abe  rdeen 

Mississippi  State  Univ 

Cornell  Aeronautical  Lab 

Waeh.  ,  D.  C, 

Reee&rch  Analysis  Corp 
STAG 

Ft.  Huachuca,  Ari* 

Ft.  Huachuca,  Arit 

Natl  Inst  of  Mental  Health 

NIH 

US  Office  of  Education 

CEIRCsrp 

Booa-Allen 

Weet,  Elec,  Corp 

VPI 

Research  Analysia  Corp 
US  Bureau  of  Budget 
Scat.  Engrg  Lab,  NBS 
GIMH ADA ,  Ft,  Belvoir 
Stanford  Univ 
Naval  Propellant  Plant 
BuShipe 

Bureau  of  Naval  Weapon* 

CEIR 

ARO-Durham 
Miasissippi  State  Univ 
Stanford  Univ 

USAE  Waterways  Experiment  Station 
Ft,  Lee,  Va. 

Picatinny  Arsenal 
Natl.  Cancer  Institute 
US  Army  Tank- Automotive  Center 
Ft,  Detrick 
F rankford  Areenal 
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Pollock,  Mr.  Abraham 
Quarles,  Dr.  Gilford  G. 

Ravenis,  Mr.  Jos.  V.  J.  ,  II 
Rhian,  Mr.  Morris  A. 
Richardson,  Mr.  B.  A. 

Rider,  Dr.  Paul  R. 

Riggs,  Mr.  Charles  W. 

Riley,  Mr.  Donald  C. 

Roberts,  Dr.  Charles 
Roberts,  Mr.  Sean  C. 

Roetzel,  Mr.  Thomas  G. 

Rhode,  Dr.  Charles  A. 
Rosenblatt,  Dr.  H.  M. 
Rosenblatt,  Dr.  Joan  R. 

Ross,  Mr.  Alan 
Roth,  Dr.  Raymond  E.  • 

Rotkin,  Mr.  Israel 
Rust,  Mr.  Philip  G. 

Sathe,  Dr.  Y. 

Schilling,  Col.  Charles  H. 
Schenker,  George 
Schmidt,  Dr.  Th.  W. 

Schalten,  Roger  W. 

Selig,  Mr.  Seymour  M. 

Selman,  Mr.  Jerome  H.  N. 
Simon,  Maj  Gen  Leslie  E.(Ret’d) 
Sirota,  Mr.  Milton 
Smith,  Dr.  Eugene  F. 

Snyder,  Mr.  Mitchell 
Soland,  Dr.  Richard 
Soni,  Mr.  Atmaram  H. 

Sorenson,  Mr.  Richard  C. 
Speckman,  Miss  Janace  A, 

Starr,  Dr.  Selig 
Stearman,  Dr.  Robert  L. 
Steedman,  Mr.  Joseph  E. 
Stephanides,  Mrs.  Agath  S. 
Sutherland,  William  H. 

Tallis,  Mr. 


OCRD,  DA 

Office,  C&ief  of  Engrg 
Johns  Hopkins  Univ 
ORG,  Edgewood 

Canadian  Army  Opera  Res  Establishment, 
Ottawa,  Canada 

Aerospace  Res  Lab,  Wright-Patterson  AFB 
Ft.  Detrick 
Amer  Stat.  Assoc 
NIH 

Mississippi  State  Univ 
Ft.  Detrick 
Johns.  Hopkins  Univ 
Bureau  of  the  Census 
Natl  Bureau  of  Standards 
Johns  Hopkins  Univ 
St.  Bonaventure  Univ 
Harry  Diamond  Labs 
Winnstead  Plantation 
Natl.  Cancer  Inst. 

US  Military  Academy,  West  Point,  N.  Y. 
Army  Weapons  Command 
US  Army  Res,  Durham 
Boeing  Co. 

Office  of  Naval  Research 
MUCOM 

Winter  Park,  Fla. 

Defense  Supply  Agency 
Waterways  Experiment  Sta. 

Univ  of  Chicago 
Research  Analysis  Corp. 

Oklahoma  State  Univ 
US  Army  Personnel  Res.  Office 
Natl  Bureau  of  Standards 
Army  Res  Office,  OCRD,  DA 
CEIR 

USA  CDCOA  Aberdeen 

US  Army  Strategy  &c  Tactics  Analysis  Gp. 
Res.  Analysis  Corp. 

Johns  Hopkins  Univ 
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Tank,  Capt.  Douglas  B. 
Tingey,  Lt.  Henry  B, 
Trethewey,  John  D. 

Tukey,  Prof.  John  W. 

Vick,  Mr.  James  A. 
VonGuerard,  Dr.  Herman  W. 
Wadley,  Dr.  F.  M. 

V  Walner,  Arthur  H. 

-  Wallach,  Mr.  Harold 
Wampler,  Roy  H. 

Watson,  Prof.  G.  S. 

Watson,  Mr.  H.  H. 
Weingaiten,  Dr.  Harry 
Weinstein,  Mr.  Joseph 
Weintraub,  Gertrude 
Wenger,  Mr.  Warren 
Westrope,  Mr.  John 
Weyland,  Maj  Carl  E. 
Wicham,  Mr.  Robert  E. 
Wigler,  Kenneth 
Wisenfeld,  Mr.  L. 

Williams,  Henry  K. 

Williams,  Mr.  Jacob  A. 
Williams,  John  N. 

Willke,  Dr.  Thomas  A. 
Wolman,  Dr.  Wm. 

Woodal,  Mrs.  Rosalie  C. 
Youden,  Dr.  W.  J. 

Young,  Mr.  Harold  W. 

Zelen,  Dr.  Marvin 
Zimmerman,  Mr.  J.  M. 
Zundel,  Brown 
Zweifel,  Mr.  Jim 


Walter  Reed  Army  Inst,  of  Res. 

Ballistic  Res  Labs 
DTC 

Princeton  Uni-v 
CRDL,  Edgewood  Arsenal 
FMC  Corp 
Ft.  Detrick 

US  Navy  Applied  Science  Lab 
Public  Housing  Adm. 

Natl.  Bureau  of  Standards 
Johns  Hopkins  Univ 

Dept  of  Natl  Defense,  Ontario,  Canada 

Dept  of  Commerce 

Ft.  Monmouth 

Picatinny  Arsenal 

Ballistic  Res  Labs,  Aberdeen 

US  Army  Strategy  &t  Tactics  Analysis  Gp. 

Hq,  OAR 

Picatinny  Arsenal 

Naval  Command  Systems  Support  Activity 
Picatinny  Arsenal 
CRDL,  Edgewood 

US  Army  Strategy  and  Tactics  Analysis  Cp. 
United  Aircraft  Corp. 

Natl  Bur  of  Standards 
NASA-Goddard  Space  Center 
HDL 

Natl  Bureau  of  Standards 

Ft.  Detrick 

Natl.  Cancer  Inst, 

Rocketdyne 

CEIR 

Natl.  Cancer  Inst. 


